extraction_base.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quelques procédures d'extraction de la base de données\n",
    "Remarque : certaines données ont été rentrées à la main car les données ont été trop irrégulières et nous n'avons que 23 pays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "import json\n",
    "liste_dict_pays=[]\n",
    "with ZipFile('north_america.zip','r') as z:\n",
    "    for i in z.namelist():\n",
    "        a=json.loads(z.read(i))\n",
    "        liste_dict_pays.append(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extraction du dictionnaire récent pour chaque pays avec wptools "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import wptools\n",
    "\n",
    "def recup(country):\n",
    "    page = wptools.page(country,silent=True)\n",
    "    page.get_parse(False)\n",
    "    return page.data['infobox']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'conventional_long_name': 'United States of America',\n",
       " 'common_name': 'the United States',\n",
       " 'image_flag': 'Flag of the United States.svg',\n",
       " 'image_coat': 'Greater coat of arms of the United States.svg',\n",
       " 'symbol_type_article': 'Great Seal of the United States#Obverse',\n",
       " 'national_motto': '<div style=\"padding-bottom:0.5em;text-align:center;\">\"[[In God We Trust]]\" </div> {{collapsible list\\n|title|=|Other traditional mottos:\\n|titlestyle|=|background:transparent;text-align:center;line-height:1.15em;\\n|liststyle|=|text-align:center;white-space:nowrap;\\n|native phrase|la|\"[[E pluribus unum]]\"|italics|=|off|ref| name=\"de facto Motto\"|{{cite web|publisher=[[U.S. Department of State]], [[Bureau of Public Affairs]] |year=2003|url= https://2009-2017.state.gov/documents/organization/27807.pdf|title=The Great Seal of the United States|accessdate=February 12, 2020}}|</ref>|rp|6, 15|<br />\"Out of many, one\"\\n|native phrase|la|\"[[Annuit cœptis]]\"|italics|=|off|ref| name=\"de facto Motto\"|rp|6, 15| <br />\"[[God|He]] has favored our undertakings\"\\n|native phrase|la|\"[[Novus ordo seclorum]]\"|italics|=|off|<br />\"New order of the ages\"}} {{native phrase|la|\"[[E pluribus unum]]\"|italics|=|off}} {{rp|6, 15}} <br />\"Out of many, one\" {{native phrase|la|\"[[Annuit cœptis]]\"|italics|=|off}} {{rp|6, 15}} <br />\"[[God|He]] has favored our undertakings\" {{native phrase|la|\"[[Novus ordo seclorum]]\"|italics|=|off}} <br />\"New order of the ages\"',\n",
       " 'national_anthem': '{{center|\"[[The Star-Spangled Banner]]\"}} <div style=\"display:inline-block;margin-top:0.4em;\"> {{center|[[File:Star Spangled Banner instrumental.ogg]]}} </div>',\n",
       " 'march': '{{center|\"[[The Stars and Stripes Forever]]\"|sfn|Kidder|Oppenheim|2007|p|=|91|ref| name=\"urluscode.house.gov\"|{{cite web |url=http://uscode.house.gov/statviewer.htm?volume=112&page=1263 |title=uscode.house.gov |author= |authorlink= |vauthors= |date=August 12, 1999 |website=Public Law 105-225 |publisher=uscode.house.gov |pages=112 Stat. 1263 |language= |quote=Section 304. \"The composition by John Philip Sousa entitled \\'The Stars and Stripes Forever\\' is the national march.\" |accessdate=September 10, 2017}}|</ref>|}} <div style=\"display:inline-block;margin-top:0.4em;\"> {{sfn|Kidder|Oppenheim|2007|p|=|91}} {{center|[[File:March, \"The Stars and Stripes Forever\" · Colonel John R. Bourgeois, Director · John Philip Sousa · United States Marine Band.ogg]]}} </div>',\n",
       " 'other_symbol': '<div style=\"padding:0.3em;\">[[File:Great Seal of the United States (obverse).svg|90px]] [[File:Great Seal of the United States (reverse).svg|90px]]</div>',\n",
       " 'other_symbol_type': '[[Great Seal of the United States|Great Seal]]:',\n",
       " 'image_map': 'USA orthographic.svg',\n",
       " 'map_width': '220px',\n",
       " 'alt_map': 'Projection of North America with the United States in green',\n",
       " 'image_map2': 'US insular areas SVG.svg',\n",
       " 'alt_map2': 'The United States and its [[Territories of the United States|territories]]',\n",
       " 'map_caption2': 'The United States, including its [[Territories of the United States|territories]]',\n",
       " 'capital': '{{plainlist|\\n* [[Washington, D.C.]]\\n* |coord|38|53|N|77|01|W|display|=|inline}} {{coord|38|53|N|77|01|W|display|=|inline}}',\n",
       " 'largest_city': '{{plainlist|\\n* [[New York City]]\\n* |coord|40|43|N|74|00|W|display|=|inline}} {{coord|40|43|N|74|00|W|display|=|inline}}',\n",
       " 'official_languages': \"{{nowrap|None at [[Federal government of the United States|federal level]]|efn|English is the [[Official language of the United States|official language]] of 32 states; English and [[Hawaiian language|Hawaiian]] are both official languages in [[Hawaii]], and English and [[Alaska Native languages|20 Indigenous languages]] are official in [[Alaska]]. [[Algonquian languages|Algonquian]], [[Cherokee language|Cherokee]], and [[Sioux language|Sioux]] are among many other official languages in Native-controlled lands throughout the country. [[French language|French]] is a ''de facto'', but unofficial, language in [[Maine]] and [[Louisiana]], while [[New Mexico]] law grants [[Spanish language|Spanish]] a special status. In five territories, English as well as one or more indigenous languages are official: [[Spanish language|Spanish]] in Puerto Rico, [[Samoan language|Samoan]] in American Samoa, [[Chamorro language|Chamorro]] in both Guam and the Northern Mariana Islands. [[Carolinian language|Carolinian]] is also an official language in the Northern Mariana Islands.|sfn|Cobarrubias|1983|p|=|195|sfn|García|2011|p|=|167}} {{efn|English is the [[Official language of the United States|official language]] of 32 states; English and [[Hawaiian language|Hawaiian]] are both official languages in [[Hawaii]], and English and [[Alaska Native languages|20 Indigenous languages]] are official in [[Alaska]]. [[Algonquian languages|Algonquian]], [[Cherokee language|Cherokee]], and [[Sioux language|Sioux]] are among many other official languages in Native-controlled lands throughout the country. [[French language|French]] is a ''de facto'', but unofficial, language in [[Maine]] and [[Louisiana]], while [[New Mexico]] law grants [[Spanish language|Spanish]] a special status. In five territories, English as well as one or more indigenous languages are official: [[Spanish language|Spanish]] in Puerto Rico, [[Samoan language|Samoan]] in American Samoa, [[Chamorro language|Chamorro]] in both Guam and the Northern Mariana Islands. [[Carolinian language|Carolinian]] is also an official language in the Northern Mariana Islands.|sfn|Cobarrubias|1983|p|=|195|sfn|García|2011|p|=|167}} {{sfn|Cobarrubias|1983|p|=|195}} {{sfn|García|2011|p|=|167}}\",\n",
       " 'languages_type': '[[National language]]',\n",
       " 'languages': '[[English language|English]]',\n",
       " 'ethnic_groups': \"{{plainlist|'''By race:'''\\n* 76.5% [[White Americans|White]]\\n* (61.1% [[Non-Hispanic whites|Non-Hispanic White]])\\n* 13.4% [[African Americans|Black]]\\n* 5.9% [[Asian Americans|Asian]]\\n* 2.7% [[Multiracial Americans|Other/multiracial]]\\n* 1.3% [[Native Americans in the United States|Native American]]\\n* 0.2% [[Pacific Islander Americans|Pacific Islander]]}} {{plainlist|'''Ethnicity:'''\\n* 18.3% [[Hispanic and Latino Americans|Hispanic or Latino]]\\n* 81.7% non-Hispanic or Latino}}\",\n",
       " 'ethnic_groups_year': '2018',\n",
       " 'demonym': '[[Americans|American]] {{efn|name|=|demonym|The historical and informal demonym [[Yankee]] has been applied to Americans, New Englanders, or northeasterners since the 18th century.}}',\n",
       " 'government_type': '[[Federalism in the United States|Federal]] [[presidential system|presidential]] [[republic|constitutional republic]]',\n",
       " 'leader_title1': '[[President of the United States|President]]',\n",
       " 'leader_name1': '{{nowrap|[[Donald Trump]]}} ([[Republican Party (United States)|R]])',\n",
       " 'leader_title2': '[[Vice President of the United States|Vice President]] {{Efn|Also president of the [[United States Senate|Senate]].}}',\n",
       " 'leader_name2': '{{nowrap|[[Mike Pence]]}} ([[Republican Party (United States)|R]])',\n",
       " 'leader_title3': '{{nowrap|[[Speaker of the United States House of Representatives|House Speaker]]}}',\n",
       " 'leader_name3': '{{nowrap|[[Nancy Pelosi]]}} ([[Democratic Party (United States)|D]])',\n",
       " 'leader_title4': '[[Chief Justice of the United States|Chief Justice]]',\n",
       " 'leader_name4': '[[John Roberts]]',\n",
       " 'legislature': '[[United States Congress|Congress]]',\n",
       " 'upper_house': '[[United States Senate|Senate]]',\n",
       " 'lower_house': '[[United States House of Representatives|House of Representatives]]',\n",
       " 'sovereignty_type': '[[American Revolution|Independence]]',\n",
       " 'sovereignty_note': 'from [[Kingdom of Great Britain|Great Britain]]',\n",
       " 'established_event1': '[[United States Declaration of Independence|Declaration]]',\n",
       " 'established_date1': 'July 4, 1776',\n",
       " 'established_event2': '{{nowrap|[[Articles of Confederation]]}}',\n",
       " 'established_date2': 'March 1, 1781',\n",
       " 'established_event3': '[[Treaty of Paris (1783)|Treaty of Paris]]',\n",
       " 'established_date3': 'September 3, 1783',\n",
       " 'established_event4': '{{nowrap|[[United States Constitution|Current constitution]]}}',\n",
       " 'established_date4': 'June 21, 1788',\n",
       " 'established_event5': '[[United States Bill of Rights|Bill of Rights]]',\n",
       " 'established_date5': 'September 25, 1789',\n",
       " 'established_event6': '{{nowrap|[[Admission to the Union|Last state admitted]]}}',\n",
       " 'established_date6': '{{nowrap|August 21, 1959 ([[Hawaii]])}}',\n",
       " 'established_event7': '[[Twenty-seventh Amendment to the United States Constitution|Last amendment]]',\n",
       " 'established_date7': 'May 5, 1992',\n",
       " 'area_link': 'Geography of the United States',\n",
       " 'area_label': 'Total area',\n",
       " 'area_footnote': '{{efn|name|=|largestcountry}}',\n",
       " 'area_rank': '3rd/4th',\n",
       " 'area_sq_mi': '3,796,742',\n",
       " 'percent_water': '6.97',\n",
       " 'area_label2': 'Total land area',\n",
       " 'area_data2': '{{convert|3,531,905|sqmi|km2|abbr|=|on}}',\n",
       " 'population_estimate': '{{increase}} 328,239,523',\n",
       " 'population_census': '308,745,538 {{efn|name|=|\"pop\"}}',\n",
       " 'population_estimate_year': '2019',\n",
       " 'population_estimate_rank': '3rd',\n",
       " 'population_census_year': '2010',\n",
       " 'population_census_rank': '3rd',\n",
       " 'population_density_sq_mi': '87',\n",
       " 'population_density_rank': '146th',\n",
       " 'GDP_PPP': '{{increase}} {{nowrap|$22.321 trillion|<!-- End nowrap. -->}}',\n",
       " 'GDP_PPP_year': '2020',\n",
       " 'GDP_PPP_rank': '2nd',\n",
       " 'GDP_PPP_per_capita': '{{increase}} $67,426',\n",
       " 'GDP_PPP_per_capita_rank': '11th',\n",
       " 'GDP_nominal': '{{increase}} {{nowrap|$22.321 trillion}}',\n",
       " 'GDP_nominal_year': '2020',\n",
       " 'GDP_nominal_rank': '1st',\n",
       " 'GDP_nominal_per_capita': '{{increase}} $67,426',\n",
       " 'GDP_nominal_per_capita_rank': '7th',\n",
       " 'Gini': '39.0',\n",
       " 'Gini_year': '2017',\n",
       " 'Gini_change': 'decrease',\n",
       " 'Gini_rank': '56th',\n",
       " 'HDI': '0.920',\n",
       " 'HDI_year': '2018',\n",
       " 'HDI_change': 'increase',\n",
       " 'HDI_rank': '15th',\n",
       " 'currency': '[[ {{#property:p38}} ]] ($)',\n",
       " 'currency_code': 'USD',\n",
       " 'utc_offset': '−4 to −12, +10, +11',\n",
       " 'utc_offset_DST': '−4 to −10 {{efn|name|=|\"time\"}}',\n",
       " 'date_format': '{{plainlist|\\n* |abbr|mm|month|/|abbr|dd|day|/|abbr|yyyy|year|\\n* |abbr|yyyy|year|-|abbr|mm|month|-|abbr|dd|day}} {{abbr|mm|month}} / {{abbr|dd|day}} / {{abbr|yyyy|year}} * {{abbr|yyyy|year}} - {{abbr|mm|month}} - {{abbr|dd|day}}',\n",
       " 'electricity': '120 V–60 Hz',\n",
       " 'drives_on': 'right {{efn|name|=|\"drive\"}}',\n",
       " 'calling_code': '[[North American Numbering Plan|+1]]',\n",
       " 'iso3166code': 'US',\n",
       " 'cctld': '; [[Generic top-level domain]]: [[.com]], [[.org]], [[.net]], [[.edu]], [[.gov]], [[.mil]]\\n; ccTLD (generally not used in the U.S.): [[.us]], [[.pr]], [[.as]], [[.gu]], [[.mp]], [[.vi]], [[.um]]'}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recup('United_States')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Affichage des clés communes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cle_commune(clé):\n",
    "    for country in liste_dict_pays:\n",
    "        if clé not in list(country.keys()):\n",
    "            return False\n",
    "    return True\n",
    "\n",
    "cles_communes=set()\n",
    "for country in liste_dict_pays:\n",
    "    for clé in list(country.keys()):\n",
    "        if cle_commune(clé):\n",
    "            cles_communes.add(clé)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'GDP_PPP',\n",
       " 'GDP_PPP_per_capita',\n",
       " 'GDP_PPP_year',\n",
       " 'GDP_nominal',\n",
       " 'GDP_nominal_per_capita',\n",
       " 'GDP_nominal_year',\n",
       " 'HDI',\n",
       " 'HDI_change',\n",
       " 'HDI_rank',\n",
       " 'HDI_year',\n",
       " 'area_rank',\n",
       " 'calling_code',\n",
       " 'capital',\n",
       " 'cctld',\n",
       " 'currency',\n",
       " 'demonym',\n",
       " 'drives_on',\n",
       " 'established_date1',\n",
       " 'established_event1',\n",
       " 'government_type',\n",
       " 'image_coat',\n",
       " 'image_flag',\n",
       " 'largest_city',\n",
       " 'leader_name1',\n",
       " 'leader_name2',\n",
       " 'leader_title1',\n",
       " 'leader_title2',\n",
       " 'legislature',\n",
       " 'national_anthem',\n",
       " 'official_languages',\n",
       " 'population_density_sq_mi',\n",
       " 'utc_offset'}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cles_communes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Affichage des exceptions pour une clé :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def exception(clé):\n",
    "    for country in liste_dict_pays:\n",
    "        if clé not in list(country.keys()):\n",
    "            print(country)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'common_name': 'Grenada', 'native_name': \"'''Grenada'''\", 'image_flag': 'Flag of Grenada.svg', 'image_coat': 'Coat of arms of Grenada.svg', 'national_motto': '\"Ever Conscious of God We Aspire, Build and Advance as One People\" {{lower|0.2em|ref|{{cite web |url=http://www.gov.gd |title=Government of Grenada Website |accessdate=2007-11-01}}|</ref>}}', 'national_anthem': \"''[[Hail Grenada]]''<br /><center>[[File:HailGrenada.ogg]]</center>\", 'royal_anthem': \"''[[God Save the Queen]]''\", 'image_map': 'Grenada in its region.svg', 'map_caption': 'Map indicating the location of Grenada in the [[Lesser Antilles]]', 'capital': \"[[St. George's, Grenada|St. George's]]\", 'coordinates': '{{Coord|12|03|N|61|45|W|type:city}}', 'largest_city': 'capital', 'official_languages': '{{unbulleted list |[[English language|English]]}}', 'regional_languages': '<small>[[Grenadian Creole English]]<br /> [[Grenadian Creole French]]</small>', 'ethnic_groups': '{{vunblist\\n  | 82% [[Afro-Grenadian|African]]\\n  | 13% [[Mulatto|Mixed]]\\n  |  2% [[Indo-Grenadians|East Indian]]\\n  |  1% Other\\n  |  1% Unspecified <small>(includes indigenous)</small>}}', 'ethnic_groups_year': '2011', 'demonym': 'Grenadian', 'government_type': '[[Unitary state|Unitary]] [[two-party system|two-party]] [[parliamentary system]] under a [[constitutional monarchy]]', 'leader_title1': '[[Monarch of Grenada|Monarch]]', 'leader_name1': '[[Elizabeth II]]', 'leader_title2': '[[Governor-General of Grenada|Governor-General]]', 'leader_name2': '[[Cécile La Grenade]]', 'leader_title3': '[[Prime Minister of Grenada|Prime Minister]]', 'leader_name3': '[[Keith Mitchell]]', 'legislature': '[[Parliament of Grenada|Parliament]]', 'upper_house': '[[Senate of Grenada|Senate]]', 'lower_house': '[[House of Representatives of Grenada|House of Representatives]]', 'sovereignty_type': 'Formation', 'established_event1': '[[Associated Statehood Act 1967|Associated State]]', 'established_date1': '3 March 1967', 'established_event2': \"'''[[Independence]]''' from the [[United Kingdom]]\", 'established_date2': '7 February 1974', 'established_event3': \"[[People's Revolutionary Government (Grenada)|Grenadian Revolution]]\", 'established_date3': '13 March 1979', 'established_event4': 'Constitution Restoration', 'established_date4': '4 December 1984', 'area_km2': '348.5', 'area_rank': '185th', 'area_sq_mi': '132.8', 'percent_water': '1.6', 'population_estimate': '{{UN_Population|Grenada}} {{UN_Population|ref}}', 'population_estimate_year': '{{UN_Population|Year}}', 'population_estimate_rank': '185th', 'population_density_km2': '318.58', 'population_density_sq_mi': '825.23', 'population_density_rank': '45th', 'GDP_PPP': '$1.801 billion', 'GDP_PPP_year': '2019', 'GDP_PPP_per_capita': '$16,604', 'GDP_nominal': '$1.249 billion', 'GDP_nominal_year': '2019', 'GDP_nominal_per_capita': '$11,518', 'HDI': '0.772', 'HDI_year': '2017', 'HDI_change': 'increase', 'HDI_rank': '75th', 'currency': '[[East Caribbean dollar]]', 'currency_code': 'XCD', 'utc_offset': '−4', 'drives_on': '[[Right- and left-hand traffic|left]]', 'calling_code': '[[+1-473]]', 'cctld': '[[.gd]]', 'footnote_a': 'Plus trace of [[Arawaks|Arawak]] {{\\\\}} [[Island Caribs|Carib]].'}\n"
     ]
    }
   ],
   "source": [
    "exception('conventional_long_name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Récupération du nom officiel à partir du dictionnaire:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_name(wp_info):\n",
    "    \n",
    "    # cas général\n",
    "    if 'conventional_long_name' in wp_info:\n",
    "        name = wp_info['conventional_long_name']        \n",
    "        return name\n",
    "    \n",
    "    #exception :\n",
    "    if 'common_name' in wp_info:\n",
    "        name = wp_info['common_name']\n",
    "        print( 'using common name {}...'.format(name),end='')\n",
    "        return name\n",
    "\n",
    "    # Aveu d'échec, on ne doit jamais se retrouver ici\n",
    "    print('Could not fetch country name {}'.format(wp_info))\n",
    "    return None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Barbados'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=recup('Barbados')\n",
    "get_name(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Récupération de la capitale :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "def get_capital(wp_info):\n",
    "    \n",
    "    # cas général\n",
    "    if 'capital' in wp_info:\n",
    "        \n",
    "        # parfois l'information récupérée comporte plusieurs lignes\n",
    "        # on remplace les retours à la ligne par un espace\n",
    "        capital = wp_info['capital'].replace('\\n',' ')\n",
    "        \n",
    "        # le nom de la capitale peut comporter des lettres, des espaces,\n",
    "        # ou l'un des caractères ',.()|- compris entre crochets [[...]]\n",
    "        m = re.match(\".*?\\[\\[([\\w\\s',(.)|-]+)\\]\\]\", capital)\n",
    "        \n",
    "        # on récupère le contenu des [[...]]\n",
    "        capital = m.group(1)\n",
    "        \n",
    "        return capital\n",
    "        \n",
    "    # Aveu d'échec, on ne doit jamais se retrouver ici\n",
    "    print(' Could not fetch country capital {}'.format(wp_info))\n",
    "    return None    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Washington, D.C.'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=recup('United_States')\n",
    "get_capital(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Récupération des coordonnées de chaque capitale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "def cv_coords(str_coords):\n",
    "    # on découpe au niveau des \"|\" \n",
    "    c = str_coords.split('|')\n",
    "\n",
    "    # on extrait la latitude en tenant compte des divers formats\n",
    "    lat = float(c.pop(0))\n",
    "    if (c[0] == 'N'):\n",
    "        c.pop(0)\n",
    "    elif ( c[0] == 'S' ):\n",
    "        lat = -lat\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 1 and c[1] == 'N' ):\n",
    "        lat += float(c.pop(0))/60\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 1 and c[1] == 'S' ):\n",
    "        lat += float(c.pop(0))/60\n",
    "        lat = -lat\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 2 and c[2] == 'N' ):\n",
    "        lat += float(c.pop(0))/60\n",
    "        lat += float(c.pop(0))/3600\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 2 and c[2] == 'S' ):\n",
    "        lat += float(c.pop(0))/60\n",
    "        lat += float(c.pop(0))/3600\n",
    "        lat = -lat\n",
    "        c.pop(0)\n",
    "\n",
    "    # on fait de même avec la longitude\n",
    "    lon = float(c.pop(0))\n",
    "    if (c[0] == 'W'):\n",
    "        lon = -lon\n",
    "        c.pop(0)\n",
    "    elif ( c[0] == 'E' ):\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 1 and c[1] == 'W' ):\n",
    "        lon += float(c.pop(0))/60\n",
    "        lon = -lon\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 1 and c[1] == 'E' ):\n",
    "        lon += float(c.pop(0))/60\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 2 and c[2] == 'W' ):\n",
    "        lon += float(c.pop(0))/60\n",
    "        lon += float(c.pop(0))/3600\n",
    "        lon = -lon\n",
    "        c.pop(0)\n",
    "    elif ( len(c) > 2 and c[2] == 'E' ):\n",
    "        lon += float(c.pop(0))/60\n",
    "        lon += float(c.pop(0))/3600\n",
    "        c.pop(0)\n",
    "    \n",
    "    # on renvoie un dictionnaire avec les deux valeurs\n",
    "    return {'lat':lat, 'lon':lon }\n",
    "\n",
    "def get_coords(wp_info):\n",
    "\n",
    "    # S'il existe des coordonnées dans l'infobox du pays\n",
    "    # (cas le plus courant)\n",
    "    if 'coordinates' in wp_info:\n",
    "\n",
    "        # (?i) - ignorecase - matche en majuscules ou en minuscules\n",
    "        # ça commence par \"{{coord\" et se poursuit avec zéro ou plusieurs\n",
    "        #   espaces suivis par une barre \"|\"\n",
    "        # après ce motif, on mémorise la chaîne la plus longue possible\n",
    "        #   ne contenant pas de },\n",
    "        # jusqu'à la première occurence de \"}}\"\n",
    "        m = re.match('(?i).*{{coord\\s*\\|([^}]*)}}', wp_info['coordinates'])\n",
    "\n",
    "        # l'expression régulière ne colle pas, on affiche la chaîne analysée pour nous aider\n",
    "        # mais c'est un aveu d'échec, on ne doit jamais se retrouver ici\n",
    "        if m == None :\n",
    "            print(' Could not parse coordinates info {}'.format(wp_info['coordinates']))\n",
    "            return None\n",
    "\n",
    "        # cf. https://en.wikipedia.org/wiki/Template:Coord#Examples\n",
    "        # on a récupère une chaîne comme :\n",
    "        # 57|18|22|N|4|27|32|W|display=title\n",
    "        # 44.112|N|87.913|W|display=title\n",
    "        # 44.112|-87.913|display=title\n",
    "        str_coords = m.group(1)\n",
    "\n",
    "        # on convertit en numérique et on renvoie\n",
    "        if str_coords[0:1] in '0123456789':\n",
    "            return cv_coords(str_coords)\n",
    "        \n",
    "    # Aveu d'échec, on ne doit jamais se retrouver ici\n",
    "    print(' Could not fetch country coordinates')\n",
    "    return None\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'lat': 45.4, 'lon': -75.66666666666667}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "e=recup('Canada')\n",
    "get_coords(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Récupération de l'HDI :"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyse préalable de l'année de HDI :"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "from zipfile import ZipFile\n",
    "import json\n",
    "\n",
    "with ZipFile('north_america.zip','r') as z:\n",
    "    for i in z.namelist():\n",
    "        i=i[:-5]\n",
    "        a=recup(i)\n",
    "        if a['HDI_year']!='2018':\n",
    "            print(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Récupération"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_hdi(wp_info):\n",
    "    a=wp_info['HDI']\n",
    "    if float(a)>=0.800:\n",
    "        a=a+' (very high)'\n",
    "    elif float(a)>=0.700 and float(a)<=0.799:\n",
    "        a=a+' (high)'\n",
    "    elif float(a)>=0.550 and float(a)<=0.699:\n",
    "        a=a+' (medium)'\n",
    "    elif float(a)>=0.350 and float(a)<=0.549:\n",
    "        a=a+' (low)'\n",
    "    return a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.795 (high)'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=recup('Panama')\n",
    "get_hdi(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Récupération des noms de drapeaux :"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Une analyse des noms de fichiers permet de voir que les noms de drapeaux commencent par le nom du pays en minuscule avec '_' comme séparateur (remplace l'espace). Exemple : 'united_states...png' pour les Etats-Unis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zipfile import ZipFile\n",
    "def get_flag(country):\n",
    "    with ZipFile('flags.zip','r') as z:\n",
    "        for i in z.namelist():\n",
    "            if country in i:\n",
    "                return i"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'united_states-190x100.png'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_flag('united_states')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "\n",
    "#\n",
    "# Ouverture d'une connexion avec la base de données\n",
    "#\n",
    "conn = sqlite3.connect('pays.sqlite')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'conventional_long_name': 'United States of America',\n",
       " 'common_name': 'the United States',\n",
       " 'image_flag': 'Flag of the United States.svg',\n",
       " 'image_coat': 'Greater coat of arms of the United States.svg',\n",
       " 'symbol_type_article': 'Great Seal of the United States#Obverse',\n",
       " 'national_motto': '<div style=\"padding-bottom:0.5em;text-align:center;\">\"[[In God We Trust]]\" </div> {{collapsible list\\n|title|=|Other traditional mottos:\\n|titlestyle|=|background:transparent;text-align:center;line-height:1.15em;\\n|liststyle|=|text-align:center;white-space:nowrap;\\n|native phrase|la|\"[[E pluribus unum]]\"|italics|=|off|ref| name=\"de facto Motto\"|{{cite web|publisher=[[U.S. Department of State]], [[Bureau of Public Affairs]] |year=2003|url= https://2009-2017.state.gov/documents/organization/27807.pdf|title=The Great Seal of the United States|accessdate=February 12, 2020}}|</ref>|rp|6, 15|<br />\"Out of many, one\"\\n|native phrase|la|\"[[Annuit cœptis]]\"|italics|=|off|ref| name=\"de facto Motto\"|rp|6, 15| <br />\"[[God|He]] has favored our undertakings\"\\n|native phrase|la|\"[[Novus ordo seclorum]]\"|italics|=|off|<br />\"New order of the ages\"}} {{native phrase|la|\"[[E pluribus unum]]\"|italics|=|off}} {{rp|6, 15}} <br />\"Out of many, one\" {{native phrase|la|\"[[Annuit cœptis]]\"|italics|=|off}} {{rp|6, 15}} <br />\"[[God|He]] has favored our undertakings\" {{native phrase|la|\"[[Novus ordo seclorum]]\"|italics|=|off}} <br />\"New order of the ages\"',\n",
       " 'national_anthem': '{{center|\"[[The Star-Spangled Banner]]\"}} <div style=\"display:inline-block;margin-top:0.4em;\"> {{center|[[File:Star Spangled Banner instrumental.ogg]]}} </div>',\n",
       " 'march': '{{center|\"[[The Stars and Stripes Forever]]\"|sfn|Kidder|Oppenheim|2007|p|=|91|ref| name=\"urluscode.house.gov\"|{{cite web |url=http://uscode.house.gov/statviewer.htm?volume=112&page=1263 |title=uscode.house.gov |author= |authorlink= |vauthors= |date=August 12, 1999 |website=Public Law 105-225 |publisher=uscode.house.gov |pages=112 Stat. 1263 |language= |quote=Section 304. \"The composition by John Philip Sousa entitled \\'The Stars and Stripes Forever\\' is the national march.\" |accessdate=September 10, 2017}}|</ref>|}} <div style=\"display:inline-block;margin-top:0.4em;\"> {{sfn|Kidder|Oppenheim|2007|p|=|91}} {{center|[[File:March, \"The Stars and Stripes Forever\" · Colonel John R. Bourgeois, Director · John Philip Sousa · United States Marine Band.ogg]]}} </div>',\n",
       " 'other_symbol': '<div style=\"padding:0.3em;\">[[File:Great Seal of the United States (obverse).svg|90px]] [[File:Great Seal of the United States (reverse).svg|90px]]</div>',\n",
       " 'other_symbol_type': '[[Great Seal of the United States|Great Seal]]:',\n",
       " 'image_map': 'USA orthographic.svg',\n",
       " 'map_width': '220px',\n",
       " 'alt_map': 'Projection of North America with the United States in green',\n",
       " 'image_map2': 'US insular areas SVG.svg',\n",
       " 'alt_map2': 'The United States and its [[Territories of the United States|territories]]',\n",
       " 'map_caption2': 'The United States, including its [[Territories of the United States|territories]]',\n",
       " 'capital': '{{plainlist|\\n* [[Washington, D.C.]]\\n* |coord|38|53|N|77|01|W|display|=|inline}} {{coord|38|53|N|77|01|W|display|=|inline}}',\n",
       " 'largest_city': '{{plainlist|\\n* [[New York City]]\\n* |coord|40|43|N|74|00|W|display|=|inline}} {{coord|40|43|N|74|00|W|display|=|inline}}',\n",
       " 'official_languages': \"{{nowrap|None at [[Federal government of the United States|federal level]]|efn|English is the [[Official language of the United States|official language]] of 32 states; English and [[Hawaiian language|Hawaiian]] are both official languages in [[Hawaii]], and English and [[Alaska Native languages|20 Indigenous languages]] are official in [[Alaska]]. [[Algonquian languages|Algonquian]], [[Cherokee language|Cherokee]], and [[Sioux language|Sioux]] are among many other official languages in Native-controlled lands throughout the country. [[French language|French]] is a ''de facto'', but unofficial, language in [[Maine]] and [[Louisiana]], while [[New Mexico]] law grants [[Spanish language|Spanish]] a special status. In five territories, English as well as one or more indigenous languages are official: [[Spanish language|Spanish]] in Puerto Rico, [[Samoan language|Samoan]] in American Samoa, [[Chamorro language|Chamorro]] in both Guam and the Northern Mariana Islands. [[Carolinian language|Carolinian]] is also an official language in the Northern Mariana Islands.|sfn|Cobarrubias|1983|p|=|195|sfn|García|2011|p|=|167}} {{efn|English is the [[Official language of the United States|official language]] of 32 states; English and [[Hawaiian language|Hawaiian]] are both official languages in [[Hawaii]], and English and [[Alaska Native languages|20 Indigenous languages]] are official in [[Alaska]]. [[Algonquian languages|Algonquian]], [[Cherokee language|Cherokee]], and [[Sioux language|Sioux]] are among many other official languages in Native-controlled lands throughout the country. [[French language|French]] is a ''de facto'', but unofficial, language in [[Maine]] and [[Louisiana]], while [[New Mexico]] law grants [[Spanish language|Spanish]] a special status. In five territories, English as well as one or more indigenous languages are official: [[Spanish language|Spanish]] in Puerto Rico, [[Samoan language|Samoan]] in American Samoa, [[Chamorro language|Chamorro]] in both Guam and the Northern Mariana Islands. [[Carolinian language|Carolinian]] is also an official language in the Northern Mariana Islands.|sfn|Cobarrubias|1983|p|=|195|sfn|García|2011|p|=|167}} {{sfn|Cobarrubias|1983|p|=|195}} {{sfn|García|2011|p|=|167}}\",\n",
       " 'languages_type': '[[National language]]',\n",
       " 'languages': '[[English language|English]]',\n",
       " 'ethnic_groups': \"{{plainlist|'''By race:'''\\n* 76.5% [[White Americans|White]]\\n* 13.4% [[African Americans|Black]]\\n* 5.9% [[Asian Americans|Asian]]\\n* 2.7% [[Multiracial Americans|Other/multiracial]]\\n* 1.3% [[Native Americans in the United States|Native American]]\\n* 0.2% [[Pacific Islander Americans|Pacific Islander]]}} {{plainlist|'''Ethnicity:'''\\n* 18.3% [[Hispanic and Latino Americans|Hispanic or Latino]]\\n* 81.7% non-Hispanic or Latino}}\",\n",
       " 'ethnic_groups_year': '2018',\n",
       " 'demonym': '[[Americans|American]] {{efn|name|=|demonym|The historical and informal demonym [[Yankee]] has been applied to Americans, New Englanders, or northeasterners since the 18th century.}}',\n",
       " 'government_type': '[[Federalism in the United States|Federal]] [[presidential system|presidential]] [[republic|constitutional republic]]',\n",
       " 'leader_title1': '[[President of the United States|President]]',\n",
       " 'leader_name1': '{{nowrap|[[Donald Trump]]}} ([[Republican Party (United States)|R]])',\n",
       " 'leader_title2': '[[Vice President of the United States|Vice President]] {{Efn|Also president of the [[United States Senate|Senate]].}}',\n",
       " 'leader_name2': '{{nowrap|[[Mike Pence]]}} ([[Republican Party (United States)|R]])',\n",
       " 'leader_title3': '{{nowrap|[[Speaker of the United States House of Representatives|House Speaker]]}}',\n",
       " 'leader_name3': '{{nowrap|[[Nancy Pelosi]]}} ([[Democratic Party (United States)|D]])',\n",
       " 'leader_title4': '[[Chief Justice of the United States|Chief Justice]]',\n",
       " 'leader_name4': '[[John Roberts]]',\n",
       " 'legislature': '[[United States Congress|Congress]]',\n",
       " 'upper_house': '[[United States Senate|Senate]]',\n",
       " 'lower_house': '[[United States House of Representatives|House of Representatives]]',\n",
       " 'sovereignty_type': '[[American Revolution|Independence]]',\n",
       " 'sovereignty_note': 'from [[Kingdom of Great Britain|Great Britain]]',\n",
       " 'established_event1': '[[United States Declaration of Independence|Declaration]]',\n",
       " 'established_date1': 'July 4, 1776',\n",
       " 'established_event2': '{{nowrap|[[Articles of Confederation]]}}',\n",
       " 'established_date2': 'March 1, 1781',\n",
       " 'established_event3': '[[Treaty of Paris (1783)|Treaty of Paris]]',\n",
       " 'established_date3': 'September 3, 1783',\n",
       " 'established_event4': '{{nowrap|[[United States Constitution|Current constitution]]}}',\n",
       " 'established_date4': 'June 21, 1788',\n",
       " 'established_event5': '[[United States Bill of Rights|Bill of Rights]]',\n",
       " 'established_date5': 'September 25, 1789',\n",
       " 'established_event6': '{{nowrap|[[Admission to the Union|Last state admitted]]}}',\n",
       " 'established_date6': '{{nowrap|August 21, 1959 ([[Hawaii]])}}',\n",
       " 'established_event7': '[[Twenty-seventh Amendment to the United States Constitution|Last amendment]]',\n",
       " 'established_date7': 'May 5, 1992',\n",
       " 'area_link': 'Geography of the United States',\n",
       " 'area_label': 'Total area',\n",
       " 'area_footnote': '{{efn|name|=|largestcountry}}',\n",
       " 'area_rank': '3rd/4th',\n",
       " 'area_sq_mi': '3,796,742',\n",
       " 'percent_water': '6.97',\n",
       " 'area_label2': 'Total land area',\n",
       " 'area_data2': '{{convert|3,531,905|sqmi|km2|abbr|=|on}}',\n",
       " 'population_estimate': '{{increase}} 328,239,523',\n",
       " 'population_census': '308,745,538 {{efn|name|=|\"pop\"}}',\n",
       " 'population_estimate_year': '2019',\n",
       " 'population_estimate_rank': '3rd',\n",
       " 'population_census_year': '2010',\n",
       " 'population_census_rank': '3rd',\n",
       " 'population_density_sq_mi': '87',\n",
       " 'population_density_rank': '146th',\n",
       " 'GDP_PPP': '{{increase}} {{nowrap|$22.321 trillion|<!-- End nowrap. -->}}',\n",
       " 'GDP_PPP_year': '2020',\n",
       " 'GDP_PPP_rank': '2nd',\n",
       " 'GDP_PPP_per_capita': '{{increase}} $67,426',\n",
       " 'GDP_PPP_per_capita_rank': '11th',\n",
       " 'GDP_nominal': '{{increase}} {{nowrap|$22.321 trillion}}',\n",
       " 'GDP_nominal_year': '2020',\n",
       " 'GDP_nominal_rank': '1st',\n",
       " 'GDP_nominal_per_capita': '{{increase}} $67,426',\n",
       " 'GDP_nominal_per_capita_rank': '7th',\n",
       " 'Gini': '39.0',\n",
       " 'Gini_year': '2017',\n",
       " 'Gini_change': 'decrease',\n",
       " 'Gini_rank': '56th',\n",
       " 'HDI': '0.920',\n",
       " 'HDI_year': '2018',\n",
       " 'HDI_change': 'increase',\n",
       " 'HDI_rank': '15th',\n",
       " 'currency': '[[ {{#property:p38}} ]] ($)',\n",
       " 'currency_code': 'USD',\n",
       " 'utc_offset': '−4 to −12, +10, +11',\n",
       " 'utc_offset_DST': '−4 to −10 {{efn|name|=|\"time\"}}',\n",
       " 'date_format': '{{plainlist|\\n* |abbr|mm|month|/|abbr|dd|day|/|abbr|yyyy|year|\\n* |abbr|yyyy|year|-|abbr|mm|month|-|abbr|dd|day}} {{abbr|mm|month}} / {{abbr|dd|day}} / {{abbr|yyyy|year}} * {{abbr|yyyy|year}} - {{abbr|mm|month}} - {{abbr|dd|day}}',\n",
       " 'electricity': '120 V–60 Hz',\n",
       " 'drives_on': 'right {{efn|name|=|\"drive\"}}',\n",
       " 'calling_code': '[[North American Numbering Plan|+1]]',\n",
       " 'iso3166code': 'US',\n",
       " 'cctld': '; [[Generic top-level domain]]: [[.com]], [[.org]], [[.net]], [[.edu]], [[.gov]], [[.mil]]\\n; ccTLD (generally not used in the U.S.): [[.us]], [[.pr]], [[.as]], [[.gu]], [[.mp]], [[.vi]], [[.um]]'}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recup('United_States')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}