Poor maintenance of cytotoxic factor expression among HIV-specific CD8+ T cells, in part caused by dysregulated expression of the transcription factor T-bet, is associated with HIV disease progression. However, the precise evolution and context in which CD8+ T cell cytotoxic functions become dysregulated in HIV infection remain unclear. Using the rhesus macaque (RM) SIV infection model, we evaluated the kinetics of SIV-specific CD8+ T cell cytolytic factor expression in peripheral blood, lymph node, spleen, and gut mucosa from early acute infection through chronic infection. We identified rapid acquisition of perforin and granzyme B expression in SIV-specific CD8+ T cells in blood, secondary lymphoid tissues and gut mucosa that collapsed rapidly during the transition to chronic infection. The evolution of this expression profile was linked to low expression of T-bet and occurred independent of epitope specificity, viral escape patterns and tissue origin. Importantly, during acute infection SIV-specific CD8+ T cells that maintained T-bet expression retained the ability to express granzyme B after stimulation, but this relationship was lost in chronic infection. Together, these data demonstrate the loss of cytolytic machinery in SIV-specific CD8+ T cells in blood and at tissue sites of viral reservoir and active replication during the transition from acute to chronic infection. This phenomenon occurs despite persistent high levels of viremia suggesting that an inability to maintain properly regulated cytotoxic T cell responses in all tissue sites enables HIV/SIV to avoid immune clearance, establish persistent viral reservoirs in lymphoid tissues and gut mucosa, and lead ultimately to immunopathogenesis and death.