Llama-2-7b-hf.txt

<s> Vaswani et al. (2017) introduced the Transformers model for sequence-to-sequence (seq2seq) tasks. It achieved state-of-the-art performance on machine translation and summarization tasks by introducing attention-based recurrent neural networks (RNNs, e.g., LSTMs) to capture long-range dependencies in the input sequence.
Transformers are composed of self-attention and feed-forward layers. The self-attention layer computes a weighted sum of inputs by attending to each input at different positions, whereas the feed-forward layer applies a linear transformation to the output of the self-attention layer. In this post, we’ll explore Transformer architectures and their training.
The architecture of a vanilla Transformer is shown in Figure 1. It consists of an encoder and a decoder. The encoder takes an input sequence $x \in \mathbb{R}^{T \times D}$ and outputs a sequence of hidden states $h_1, h_2, \hdots, h_T \in \mathbb{R}^{D}$. The decoder takes the hidden states of the encoder and outputs a sequence of hidden states $d_1, d_2, \hdots, d_T \in \mathbb{R}^{D}$.
Figure 1: Architecture of a vanilla Transformer.
The encoder is a stack of $N$ identical layers, each of which consists of a multi-head self-attention layer and a feed-forward layer. The multi-head self-attention layer has $K$ heads, each of which computes a weighted sum of the input sequence using different sets of keys and/or values. The feed-forward layer applies a linear transformation to the output of the self-attention layer.
The decoder is a stack of $N$ identical layers, each of which consists of a multi-head self-attention layer and a feed-forward layer. The multi-head self-attention layer has $K$ heads, each of which computes a weighted sum of the input sequence using different sets of keys and/or values. The feed-forward layer applies a linear transformation to the output of the self-attention layer.
The input sequence is fed into the encoder and the output sequence is fed into the decoder. The two sequences are concatenated and fed into the final feed-forward layer to compute the output sequence.
Training a Transformer
The objective function of a Transformer is to minimize the cross-entropy loss between the ground truth and the predicted output sequence. The cross-entropy loss is computed as follows:
$$\begin{align} L &= - \sum_{i=1}^{T} \log p(d_i | x) \\ &= - \sum_{i=1}^{T} \log \frac{e^{v_i}}{\sum_{j=1}^{D} e^{v_j}} \end{align}$$
where $p(d_i | x)$ is the probability of the $i$-th element of the output sequence being $d_i$, $v_i$ is the $i$-th element of the output sequence, and $e^{v_i}$ is the element-wise exponential of $v_i$.
During training, the encoder and decoder are jointly trained by backpropagating the gradients of the cross-entropy loss through the network. This is known as end-to-end training.
Vaswani, A., Uszkoreit, J., Gomez, A. N., Lopez, Cañibano, A., Matena, J., … Devlin, J. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762.
Khandelwal, S., Choudhury, B., Srikanta Reddy, K., & Gupta, A. (2018). Neural Turing Machines: A Survey. IEEE Transactions on Neural Networks and Learning Systems, 29(12), 3606–3621. https://doi.org/10.1109/TNNLS.2018.2860041
Bengio, y., Courville, A., Fournier-Viger, P., Grangier, D., Hénríquez, N., Léonard, J., … Rousseau, D. (2015). Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8), 1890–1917. https://doiforce.acm.org/10.1109/TPAMI.2014.2324898
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780. https://doi.pionline.org/10.1162/neco.1997.9.8.1735
Graves, A., Wayne, G., Danihelka, I., & Gulcehre, C. (2014). Speech Recognition with Deep Neural Networks. ICASSP, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6735–6743. https://doi.org/10.1109/ICASSP.2014.6858894
Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25, 1097–1105. https://doi.org/10.1145/2383536.2383692
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Hubbard, W., Jackel, L. D., … Wojna, Z. (1989). Handwritten Digit Recognition Using a Back-Propagation Network. Neural Computation, 1(4), 541–551. https://doi.pionline.org/10.1162/neco.1989.1.4.541
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., … Rabinovich, A. (2015). Going Deeper with Convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, 1–9. https://doi.org/10.1109/CVPR.2015.7299984
Bahdanau, D., Cho, K., Bougares, F., Schwenk, H., Escorcia Romero, J., Joulin, A., … Ammar, W. (2014). Towards End-to-End Speech Recognition. In International Conference on Spoken Language Processing (ICSLP), 1533–1538. https://doi.org/10.21437/Interspeech.2014-1201
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K. Q., Welling, M., Cross-Grove, A., … Tancik, R. (2017). Densely Connected Convolutional Networks. In International Conference on Learning Representations (ICLR), 1–10. https://openreview.net/forum?id=BWwD5b3JpbmF0aW9uX3R5cGU9dHJ1ZS5jb20=.
Han, J., Mao, Z., Zhao, X., Liu, Z., and He, K. (2015). Deep Learning in Natural Language Processing. Foundations and Trends in Information Retrieval, 9(3–4), 207–273. https://doi.org/10.1561/2400000027/
Sennrich, R., Haddow, B., Birch, A., Bradbury, J., and Nagelmeier, M. (2016). Neural Semantic Parsing. In 2016 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), 1621–1631. https://www.aclweb.org/anthology/D16-1102/
Mikolov, T., Karafiát, M., Burget, L., Červonenkov, V., Corneil, D., Shazeer, N., and Khudanpur, S. (2010). Efficient Estimation of Word Vectors in Large Corpora. In 2010 Conference on Empirical Approaches to MT (EAMT), 147–154. https://www.aaai.org/ojs/index.php/aimag/article/view/1008/1000
Hermann, N., Dahl, J., Luduschenko, A., Schick, B., Niekrasz, D., and Andor, N. (2015). Neural Language Models Reveal Insights into Human Memory. Nature Neuroscience, 18(10), 1596–1604. https://doi.org/10.1038/nn.4100
Bowman, S., Farhadi, A., Girshick, R., Kavukcuoglu, K., Markoulaki, E., Suleyman, H., Torralba, A., Zemel, R., and Donahue, J. (2015). Arbitrary Relation Extraction with Conditional Random Fields. In 2015 Conference on Neural Information Processing Systems (NIPS), 2762–2770. https://papers.nips.cc/paper/5636-arbitrary-relation-extraction-with-conditional-random-fields.pdf
Bordes, A., Collobert, R., Weston, J., and Welinder, P. (2014). Translating Rules into Structure for Text Understanding. In 2014 Conference on Artificial Intelligence (AAAI), Palo Alto, CA, USA, 2325–2331. https://dl.acm.org/citation.cfm?id=2569955.2569960
Colmenarejo, S., Grefenstette, G., Hadsell, R., and Quattoni, A. (2016). Generative Adversarial Networks for Image Synthesis with Photorealistic Quality. CoRR abs/1606.08929. http://arxiv.org/abs/1606.08929
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Rumelhart, D., and Frey, B. (2014). Generative Adversarial Nets. CoRR abs/1411.1789. http://arxiv.org/abs/1411.1789
Kingma, D. and Ba, J. (2014). Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980. http://jmlr.org/papers/volume15/14-698/14-698.pdf
Krishnaswamy, S., Agarwal, A., and Jaitly, N. (2017). Deep Contextual Bandits. CoRR abs/1703.00505. http://arxiv.org/abs/1703.00505
Lillicrap, T., Hodgkin, G., Burnton, S., Ramachandran, R., Beattie, A., Sadik, A., Antol, R., Ardemeani Ashtiani, S., Charvalos, M., Fadigas, A., Flamari, S., Ghassemipour Kouchesfehani, M., Goyal Punjabi, M., Hejrati Moghaddam, S., Iyyer, V., Kalashnikov, I., Karkanak, A., Kolotovskyy, V., Laidlaw, D., Latif Qureshi, M., Leibs, J., Levine, S., Liu, T., Lucey, K., Machavarov, V., Manning, R., McAllister, A., Melgarejo, J., Metz, M., Mishkin Emmett, E., Montemerlo, M., Neelakantan, A., Ng, A., Osindero, O., Pastor Pueblas, P., Precup, D., Rajeswaran, A., Robbins, J., Rusu, A., Sanchez Mueggler, M., Sastry, S., Shalev-Shwartz, S., Silver, D., Simonyan, Kvashnin, A., Solomon, J., Stentzschi, T., Steunebrink, M., Sundararajan, S., Szewczyk, A., Tangkawattana, S., Tejani Shakkottai, V., Theobald, N., Thickstun, J., Vezhnevets, A., Vladimirov Petrov, A., Wierstra, D., Wolski, P., Xiong, Z., Yamashita Anjyo, K., Yeom, H., Yushanov, I., Zhang, Y., Zoigman Bell, J., Abbeel, P., Acharya, A., Adlakha, S., Ajjanagadde, S., Alvarez Delgado, F., Amodei, D., Amos, B., Anandkumar, A.Ћ, Arora, S., Asgari, S., Atanasov, P., Babuschkin, I., Bachlechner, R., Bagnell, A., Baglietto di Villa Giusti, M., Balduzzi, D., Banerjee, A., Barrault, L., Basu Bhattacharya, S., Battaglia, P., Baydachnko, D., Beaulieu, J., Bellegueule, B., Benhamou, S., Berkenkamp, K., Bertin-Mahieux, F., Bidgoli, M., Biggio, B., Billingsley, J., Bindemann, M., Bird, R., Blaauwgeers, B., Blunsom, P., Bockholt, U., Bojarski, M., Bolz, J., Borgwardt, K., Botvinik-Nezer, O., Boureau, Y., Bourlard, H., Bredereck, C., Brehmer, J., Brendel, W., Brockschnieders, M., Brugger, P., Brysbaert, M., Budden, E., Buesing, L., Buenaventura, J., Buhmann, J., Bulling, R., Buslai, V., Buttgenpütz, T., Bychkovsky, I., Caballero-Gaudes, C., Cadamuro, G., Caicedo, J., Caliskan, A., Callaghan, G., Cangelosi, A., Caponnetto, P., Carlevaro, D., Carlsson, S., Casanellas, A., Castañeda-Muñoz, R., Catenacci, D., Cavallaro, J., Cavallo, L., Cerisin, G., Cesarini Decarli, C., Chakrabarti, D., Chapuis-Lardy, L., Chaundhuri, A., Chebotar, L., Cherry, E., Chintala, S., Chopra, S., Christodoulou, G., Churchill, E., Clausner, T., Clifton, C., Cohen Gurshtein, T., Cole-Turner, R., Collomosse, J., Combescure, C., Conneau, A., Contestabile, M., Coorssen, J., Cortes, A., Cottatellucci, L., Cowan, J., Craighero, P., Creemers Peeters, A., Cricklewood, G., Crookshank, J., Cruciani Vulcani, F., Cuayáhuitl, R., Cziborraga, G., Daelemans, W., Dalla Bella Salamon, F., Damianou, A., Darwazeh, G., Dehghani Sharifabadi, A., Demirörs, U., Dengg, J., Derntl, M., Deselaers, T., Develivier, D., Di Carpegna Falasconi, T., Dieleman, J., Díaz Pernas, M., Doersch, C., Dorrestein Akkerhuis, M., Dovgopol, V., Dreiseitl, S., Dujardin, J., Dunning, E., Duran Nieto, J., Edelman, Z., Efros, A., El Yacoubi, B., Ellis, D., Engelbrecht-Wolfslag van Loon, M., Erkut, S., Escalera Scolfaro, L., Estivill-Castroverde, C., Etessami, A., Evgeniadou, E., Faber, P., Fancello, L., Feijóo Alcaide, A., Ferragina, P., Fernandez-Ruiz Joven, J., Fischbach Kranzlmüller, D., Fleck, J., Floris, V., Fokkink, R., Fonseca-Pedrero, E., Forlizzi, J., Fortunato, S., Francheschi, M., Franssen, M., Freudenthaler, H., Friederich, H., Frojmovic, J., Funke, K., Galassi, D., Garcia-Sanchez, J., Gaschler-Markefka, B., Gawehn, J., Gentil-Beccot, A., Gervinka, M., Geyer-Schulz, A., Giannotti, F., Gil-Dubbeldam, M., Gippius, A., Girod, B., Glauner, M., Glöckner, F., Goebel, R., Golovchenko, Javadov, R., Gomes, J., Gonçalves, P., Gonzalez de Cosio Triviño, I., Goossens, L., Gorin, A., Gräf, C., Greuel, T., Grujic, D., Gschwandtner, M., Güçlü, U., Guerrouj, M., Gutierrez Nájera, J., Haasdonk, T., Hadzihasanovic, M., Hahnloser, R., Hallensleben, N., Hamalainen, R., Hanneke, D., Hansen, T., Haraldsen, K., Hardmeier, T., Hartel, P., Haschka, M., Haussmann, A., Hawranek, P., Haynatzki, K., Heckel, R., Hendriks, W., Henze, M., Herghelegiu, S., Herrschel, D., Herscheid, M., Heymann, P., Hildebrandt, G., Hilger, T., Hillenkamp, F., Hirschmugl, C., Hoefelmeyer, M., Hoffmann, T., Hofestädt, R., Holtkamp Wibier, J., Homolka, K., Hörtnagl, H., Horvath, I., Houzévidou, T., Hrabovsky, J., Hruška, J., Hugenschmidt, C., Huisse, B., Huitema, D., Hummelgård, H., Iacobacci, G., Ibáñez Llorente, S., Idczak, R., Ilieva, Y., Incerti, S., Ingoldbeen, M., Ivanenko, Y., Izaguirre Martinez, J., Jakobs, K., Jalobeanu, R., Jamieson, P., Januszkiewicz, L., Jasinski, M., Jaspers, M., Jeanneret, J., Jeremie, H., Jiménez Bermúdez, J., Joergenschläger, J., Johansson, J., John, M., Jonckheere, A., Jongenelis, I., Jordan, K., Juillaguet, S., Kabuš, P., Kachelrieß, M., Kaempfer, B., Kalogeropoulos, A., Kammerer, E., Kanellopoulos, K., Kapusta, F., Karavakis, E., Karlsson Thunqvist, P., Kasieczka, Ł., Kazantzidis, S., Kelich-Konarska, M., Kenyon, I., Kersevan, B., Khachatryan, G., Kippenhahn, R., Kisiel de Rozanski, M., Knobel, M., Kochertritthalahtu, P., Kolb, U., Komorowska, M., Kondrashov, V., Konoplya, E., Kopernik, J., Korol, I., Kosmopoulos, M., Kotovsky, J., Koutchouk, J.-P., Kraus, M., Kreykenbohm, I., Krieger, P., Krüger, H., Kudryavtsev, V., Kuijken Koerts, J., Kurfürst, C., Kushnir, D., Lacroix, R., Lagoutte, D., Lambourne, R., Lambrechts, L., Landgraf, J., Langenbucher, A., Lathuilière, M., Lauscher, M., Lazzeri, M., Leblond, L., Leccacorvi, F., Lee, S.-H., Leonidopoulos, C., Leontsinis, S., Li, B., Libanati, S., Lindquist, B., Lindegren, L., Linné, E., Lo Presti, D., Lopes da Silva, J., Lorenzon, M., Losurdo Giusberti, L., Lubashevskiy, A., Lucarelli, F., Luci, C., Lucio-Patron, J., Luminari, L., Lyakhovitsky, V., Maccarrone, G., Maggiora, L., Maev, O., Mainzdermayer, I., Makarov, D., Malbrán, Máñez, L., Mandat Défossez, D., Manfredi, Pafundi, R., Mansouli, E., Marangotto, D., Marchionni, A., Margutti, R., Marka, Z., Martin, J.-P., Martynov, D., Massimiliano, M., Matteuzzi Cinquegrani, L., Maugeri, E., Mazzaferro, A., Mazzucato, A., Mele, S., Melissari, M., Menichelli, M., Mereghetti, S., Messina, S., Metcalf, W., Michelson, P., Mihăiescu, B., Milanesio, M., Minervini, A., Mirzoyan, R., Mitsou, A., Moderski, R., Molinari, P., Monterubbianesi, L., Montesano, S., Montezemolo, L., Morandin, M., Moreau Salanave, B., Mostefaoui, A., Motta Trombetta, M., Mróz, P., Muccifora, V., Muleri, P., Murphy, S., Musumeci, M., Myagkov, A., Nadal-Muller, M., Nandakumar, R., Nardecchia, I., Naumann-Godo, M., Negri Altomare, F., Nelemans, G., Nevski, P., Nicastri, A., Nicolaidou, R., Nikolenko, S., Nissanke, M., Novikov, D., O'Brien, P., Odesskii, D., Olchawski, M., Onofre, A., Orlandini, M., Ostrovidov, A., Ozben, C., Palazzi, P., Panagouliotis, C., Paredes Huerta, J., Parsons, A., Pascoli, D., Passuello, D., Patricelli, S., Pavlidou, V., Pech, M., Pelayo, R., Perrin, E., Persic, M., Petitjean, O., Piacentini Rossi, E., Pilia, M., Pilkington, K., Pinar Arpacıoğlu, Ö., Piron, F., Pivovaroff, M., Plagnol, E., Podsiadlowski, Ph., Polcaro, Vincenzo, Porceddu, S., Postiglione, F., Pozanenko, A., Predazzi, E., Predehl, Ph., Prieto, J., Procaccini, R., Proudlove, S., Pulliam, C., Punturo, M., Quarrie, D., Radovic, M., Raffelt, G., Rahoui, F., Raimondi, L., Rapin, D., Rasheed, B., Ravenscroft, T., Redaelli, M., Reig, P., Renucci, T., Resconi, S., Ribolzi, G., Richichi, A., Ricci, F., Riggi, F., Riitters, M., Robba, R., Rodighiero, G., Roehl, W., Rolland, L., Romagnoli, G., Ros, E., Rossignol, M., Rouvinet, J., Roussel-Dupre, R., Rubbia, A., Rudak, B., Rybicki, K., Sacco, R., Sagrista, M., Sakellariadou, M., Salatiello Filippone, H., Salvati Marcon, M., Sanchez Mayordomo, C., Santangelo, A., Sarazin, C., Savchenko, V., Schipani, E., Sciortino Soffitta, U., Scodellaro, L., Seifert, F., Semennikov, A., Servoli Lombardi, L., Shaposhnikov, M., Shears, T., Sherwood, P., Sillou, P., Simi, G., Skidmore, J., Slutsky, J., Smadja, G., Snow, G., Soumaintraveler, C., Spaan, B., Spradlin, P., Stanco-Bertrand, A., Stamerra, A., Starodubtsev, A., Steinbrück, G., Stracka, S., Straticiuc, M., Sutantawibul, S., Swientek, K., Szczekowski, M., T'Jampens, S., Talby, M., Tarabrin, Sipahigil, S., Tavernier, S., Taylor, G., Teixeira-Dias, P., Terreni, M., Testa, M., Theveniau-Pelzer, T., Thomé, C., Thonhauser, A., Torróntegui, A., Tournefier, E., Trabelsi, K., Tran, M., Trisovszky, E., Tsaregorodtsev, A., Ueda, I., Ukleja, A., Ungerbäck, D., Usai, G., Valenti, G., Vannini, C., Varagnolo, M., Vecchi, M., Ventura Chiaradio, M., Veres, G., Verdier, R., Vernet-Maury, E., Vertogradov, L., Vicini, A., Villanueva Crespo, A., Vorobyev, A., Wackerle, F., Walck, C., Wardrope, D., Wasem, J., Watanabe, K., Weidenkaff, P., Wendland, L., Wester, W., Wilkinson, G., Wilson, F., Wislicki, W., Wyllie, K., Yagil, A., Yamamoto, T., Yang, Y., Younus, M., Yuksel, S., Zacek, V., Zachariadou, K., Zalesskii, A., Zanettin, M., Zejma, J., Zer-Zion, D., Zimmermann, R., Zioutas, K., Zurawski, P., Žurek, M., ÄŒervenkov, Ä�., čermák, V., Aad, G., Abbaneo, D., Abdallah, J., Abdullin, S., Abramowicz, H., Abreu, P., Acharya, B., Adamczyk, L., Adelman, J., Adomeit, S., Affolder, A., Agapitos, W., Ahlen, S., Ahmad Al Mahmood, M., Ainouz, D., Ajaltouni, Z., Akimov, A., Alberghi, Gérard, Aleksan, R., Alessandro, G., Alexakhin, R., Alexandre, G., Alezri, M., Allaby, J., Allekotte, I., Almalaq, A., Aloisio, A., Alon, E., Alsaihati, A., Altae-Tran, The, Altinuk, M., Alves Garcia, P., Amadio, L., Ameri, A., Amor Dos Santos, A., Anastasiou, G., Anderhub, H., Angelidakis, S., Angerami, A., Annovi, A., Antinori, F., Antonelli, M., Antonioli Severini, F., Aperio Bella, L., Aquines Gutierrez, O., Archilli, F., Argyropoulos, S., Armbruster, A., Arratia Torres, M., Artamonov, A., Artecone, S., Aslanides, E., Assamagan, K., Astbury, J., Atac, R., Audurier, L., Aurisano, A., Avakian, H., Azuelos, G., Baalouch, M., Bachmair, F., Bachynsky, M., Backovic, S., Badalov, A., Baesso, C., Baldini, W., Balli, F., Ballestrero, M., Baltayian, C., Bamoulidis, G., Banas, E., Bank, M., Barbaglieri, M., Barberio, E., Baroncelli, A., Barreiro Rodriguez, P., Bartalini, P., Bassompierre de Boissoudy, Q., Bastid, N., Batley, R., Bauerdick, L., Bazergui, M., Bechtel, M., Beck, H.Ъ, Beddow, T., Begalli, M., Behrendt, R., Bellagamba, L., Bellettini, G., Belogurov, S., Benayoun, M., Bentvelsen, S., Berceanu, I., Berezhnoy, A., Bernet, R., Bertolucci, A., Besancon, M., Besson, N., Bethani, I., Bhattacharya, S., Bianchi, F., Bianchini, L., Bien, A., Bifani, S., Bilandzic, M., Billowes, J., Birnholz, J., Bizouard, M., Black, J., Blair, R., Blanchard, P., Blocker, C., Blondel, A., Bobbink, G., Bocci, V., Bohm, P., Bokulich, K., Bondar, A., Bonivento, W., Borghi, S., Borisov, A., Boscherini, D., Botelho-Dow, J., Boudreau, J., Bowcock, T., Boxer, D., Bozson, W., Braccini, S., Bradl, M., Brandt, S., Bridgeman, A., Britsch, M., Brodzicka, J., Brook, N., Brown, R., Buchmuller, O., Buck, C., Budagov, J., Bulanov, S., Burger, J., Burns, D., Busenitz, J., Buttar, C., Buytaert, J., Cabrillo, J., Cacciapaglia, F., Calderini, G., Campanelli, M., Camplani, A., Capiluppi, P., Cara Romeo, G., Cardinale, R., Carolus, D., Cartwright, J., Casadei, D., Cassel, D., Castello, R., Cattai, A., Cauet, C., Cavalli-Sforza, M., Cavoto, G., Ceccucci, A., Centonze, V., Cerutti, F., Certik, O., Chaloupkova, R., Charpentier, P., Chatterjee, A., Chemarin, M.ardi, Chepurnov, A., Chiapolini, N., Chrzaszcz, M., Ciocca, C., Cindolo, F., Citron, Z., Clarke, P., Clemencic, M., Cliff, H., Cobal, M., Coco, V., Colijn, A., Colombi, S., Colon, D., Condemarín, M., Contreras, J. L., Contri, R., Conventi, F., Consonni, G., Constantinescu, S., Conversi, L., Couchot Fraipont, G., Couyoumtzelis, C., Covarelli, R., Cozzolino, A., Craik, D., Crede, A., Cristinziani, M., Crocombe, A., Crosetti, G., Cruz Torres, M., Crystal-Miranda, M., Cunliffe, S., Currie, R., Curtil, S., Cybulski, M., D'Ambrosio, C., Dabrowski, W., Dasgupta, A., Davidek, T., Dawson, I., De Bruyn, K., De Capua, S., De Cian, M., De Miranda, J. M., De Paula, L., De Salvo, A., De Visscher, S., De Wolf, E., Del Buono, L., Deliot, L., Denes, P., Derkach, D., Detraz, S., Diamanti, P., Di Gregorio, M., Di Lodovico, F., Dijkstra, H., Djambazov, L., Dobos, D., Doglioni, C., Dolbeau, J., Donadoni, C., Drago, E., Drisko, C., Dupertuis, F., Duric, S., Dutta Majumdar, A., Dyndal, M., Easo, S., Egede, U., Eisenhardt, S., Eitschberger, U., El Rifai, I., Elsasser, C. M., Engler, J., Enqvist, K., Ent, F., Erdmann, J., Erfani, M., Escudero, A., Esposito, G., Esteban Fuertes, J., Evans, H., Falabella de Oliveira, A., Fassnacht, P., Favara, A., Fedin, O., Felcini, M., Ferguson Tuttle, T., Ferrer-Ribas, E., Ferro-Luzzi, M., Ferrari, P., Filthaut, F., Finocchiaro, G., Fitzpatrick, A., Fontanesi, E., Ford, W. T., Formanek, T., Forty, R., Frank, M., Franzini, A., Fray, N., Frey, A., Friis, E., Fromm, M., Fujiwara, M., Furfaro, E., Gallas Torrejón, J., Galli Della Loggia, G., Ganguli, S., Garofoli, J., Gavrilenko, I., Gay Ducimeti, S., Geich-Gimbel, C., Gemmeke, D., Genest, M.-O., Geraldes Iturbe, J., Ghosh, S., Gillberg, J., Gingrich, D., Girò, F., Giuliani, A., Goldfarb, S., Golonka, J., Gómez-Ceballos, J., Gopalakrishnan, G., Gordon Howarth, S., Gotti Moscardini, E., Grabowska-Boldyreff, M., Grancagnolo, F., Graugès, E., Grassi Vittadini, G., Greening, E., Griffioen, K., Grosdidier, G., Grunhaus, J., Guaraldo, N., Guidoboni, G., Guilbaud, O., Gunthorpe, D., Gutsche, J., Habib Najafi, M., Hajduk, M., Halkiadakis, E., Haller, J., Hamacher, K., Hanemaayer, J., Hansmann-Menzemer, S., Harnew, N., Harrison, J., Hartmann, T., Hatagozaki, H., Hattori, K., Haxton, W., Hayashi, T., Hazou, A., Hedberg, Viggo, Helgesson, T., Hemingway, R., Henderson, R., Henrard, P., Hernández-Pajares, M., Herndl, R., Hertenbeller, P., Heusser, G., Hippolyte, B., Hoballah, M., Hoeferkamp, M., Hoenig, J., Holzbauer, J., Hoppmann, C., Hornbostel, A., Hossain, M., Hotchkis, M., Howe, P., Hughes, G., Hultqvist, Kärstin, Hurlburt, N., Hussein, M., Hynds, D., Ichikawa, Y., Ikeda, Y., Illarionov, A., Imbierowicz, K., Indelicato, P., Isidori, G., Ishikawa, S., Itow, Y., Iwasaki, H., Iwashita, M., Jackson, J., Jacquemot, S., Jaeger, H., Jakobsson, U., Jain, A., Janssen, J., Jarlskog, C., Jayatilaka, B., Jean-Marie, L., Jenni, P., Jiang, P., Jingjitsawee, W., Jones, G., Jung, H., Junquera, J., Junk, T., Kadastik, M., Kadojewski, H., Kajomovitz, E., Kalinikos, K., Kamae, T., Kameda, J., Kamyshkov, Yu., Kanazawa, N., Kangaroo, A., Kanzaki, J., Kaplan, D., Karlen, D., Karpov, S., Katayama, T., Katsuragawa, Y., Kaufmann, G., Kephart, T., Ketelhut, S., Khan, A., Kim, J., Kindlmann, T., Kiryunov, S., Kitagaki, H., Klappert, K., Klimentov, A., Kobayashi, T., Kobayashi, T., Kocheni, S., Kodama, K., Kogan, V., Kole, B., Komamiya, S., Kongasund, A., Konstantinidis, N., Kowalski, S., Kramer, M., Kravtsov, P., Krofcheck, D., Krueger, H., Kubodera, T., Kucharczyk, M., Kulikov, A., Kumakubo, T., Kunugi, T., Kurochkin, V., Kuzminov, V., Kyberd, P., La Thi, V., Lauret, J., Layoun, T., Lecompte, T., Lees, J., Leggett, R., Lenzen, G., Leroy, O., Levchenko, S., Levinson, L., Li, G., Liberati, S., Lichard, P., Lindroos, M., Liu, H., Livan, M., Lobodzinski, A., Locci, E., Lokajicek, M., Longhin, A., Lopez, C., Lourenco, C., Loveluck, R., Lowenstein, B., Luetticke, R., Luparello, G., Lynch, G., Maeshima, K., Mahapatra, D., Majerotto, P., Malaescu, B., Malinin, A., Malyshev, V., Mammini, R., Mangiarotti, F., Mannheim, K., Manousakis-Katsikakis, A., Marcinkowski, P., Marin Benito, C., Marrodan Undagoitia, M., Masciovecchio, M., Mastrogiacomo, F., Matthews, J., Mawhinney, B., Maxfield, S., Mayergoyz, D., McCubbin, M., McFarlane, K., McKenna, J., McNab, A., Medinaceli, E., Mekhfi, C., Melkumov, P., Mengel, F., Menshikov, A., Mereminskiy, E., Mesropian, C., Meyer, J., Michaely, T., Mikocki, M., Milner-Cochet, H., Minaenko, A., Miquel, R., Mitrovski, M., Moa, T., Mockenroth, R., Mohanty, B., Molina Ramos, J., Monteil, S., Montalbano, A., Moore, R., Mori Becerra, R., Morozov, D., Moser, H., Moukhtar, M., Movilla Fernandez, M., Mulder, D., Munhoz, M., Murat, P