The developpment of high throughput single-cell technologies now allows the investigation of the genome-wide diversity of transcription. This diversity has shown two faces : the expression dynamics (gene to gene variability) can be quantified more accurately, thanks to the measurement of lowly-expressed genes. Second, the cell-to-cell variability is high, with a low proportion of cells expressing the same gene at the same time/level. Those emerging patterns appear to be very challenging from the statistical point of view, especially to represent and to provide a summarized view of single-cell expression data. PCA is one of the most powerful framework to provide a suitable representation of high dimensional datasets, by searching for new axis catching the most variability in the data. Unfortunately, classical PCA is based on Euclidian distances and projections that work poorly in presence of over-dispersed counts that show zero-inflation. We propose a probabilistic PCA for single-cell expression data, that relies on a sparse Gamma-Poisson model. This hierarchical model is inferred using a variational EM algorithm, and we revisit the selection of the number of axis using an integrated likelihood criterion. We show how this probabilistic framework induces a geometry that is suitable for single-cell data, and produces a compression of the data that is very powerful for clustering purposes. Our method is competed to other standard representation methods like tSNE, and we illustrate its performance on a project that is based on transcriptomic data of CD8+ T cells. Understanding the mechanisms of an adaptive immune response is of great interest for the creation of new vaccines. We show that our method allows a better understanding of the transcriptomic diversity of T cells, which constitutes a new challenge to better characterize the short and long-term response to vaccination.