A continuum damage model is developed for the linear viscoelastic behavior of composites with microcracks consisting of an isotropic matrix reinforced by two arbitrarily independent and inextensible fiber families. Despite the fact that the matrix material is isotropic, the model in consideration bears the characteristic of directed media included in the transverse isotropy symmetry group solely due to its fibers distributions and the existence of microcracks. Using the basic laws of continuum damage mechanics and equations belonging to kinematics and deformation geometries of fibers, the constitutive functions have been obtained. It has been detected as a result of the thermodynamic constraints that the stress potential function is dependent on two symmetric tensors and two vectors, whereas the dissipative stress function is dependent on four symmetric tensors and two vectors. To determine arguments of the constitutive functionals, findings relating to the theory of invariants have been used as a method because of the fact that isotropy constraint is imposed on the material. As a result the linear constitutive equations of elastic stress, dissipative stress, and strain energy density release rate have been written in terms of material coordinate description. Using these expressions, total stress has been found.