fth2023-model-deployment.ipynb

{"cells":[{"cell_type":"code","execution_count":2,"metadata":{},"outputs":[],"source":["import json\n","            \n","with open('env_var.json') as f:  \n","    CFG = json.load(f)"]},{"attachments":{},"cell_type":"markdown","metadata":{},"source":["# Deploying to Sagemaker through HuggingFace Models"]},{"cell_type":"code","execution_count":3,"metadata":{"execution":{"iopub.execute_input":"2023-01-11T13:30:17.173936Z","iopub.status.busy":"2023-01-11T13:30:17.173564Z","iopub.status.idle":"2023-01-11T13:30:32.907142Z","shell.execute_reply":"2023-01-11T13:30:32.905853Z","shell.execute_reply.started":"2023-01-11T13:30:17.173906Z"},"trusted":true},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: sagemaker in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (2.128.0)"]},{"name":"stderr","output_type":"stream","text":["\n","[notice] A new release of pip available: 22.2.2 -> 22.3.1\n","[notice] To update, run: python.exe -m pip install --upgrade pip\n"]},{"name":"stdout","output_type":"stream","text":["\n","Requirement already satisfied: protobuf<4.0,>=3.1 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (3.15.6)\n","Requirement already satisfied: smdebug-rulesconfig==1.0.1 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (1.0.1)\n","Requirement already satisfied: pandas in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (1.2.3)\n","Requirement already satisfied: numpy<2.0,>=1.9.0 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (1.21.6)\n","Requirement already satisfied: boto3<2.0,>=1.26.28 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (1.26.47)\n","Requirement already satisfied: importlib-metadata<5.0,>=1.4.0 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (3.7.3)\n","Requirement already satisfied: google-pasta in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (0.2.0)\n","Requirement already satisfied: protobuf3-to-dict<1.0,>=0.1.5 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (0.1.5)\n","Requirement already satisfied: packaging>=20.0 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (20.9)\n","Requirement already satisfied: attrs<23,>=20.3.0 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (22.2.0)\n","Requirement already satisfied: pathos in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (0.3.0)\n","Requirement already satisfied: schema in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from sagemaker) (0.7.5)\n","Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from boto3<2.0,>=1.26.28->sagemaker) (0.6.0)\n","Requirement already satisfied: botocore<1.30.0,>=1.29.47 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from boto3<2.0,>=1.26.28->sagemaker) (1.29.47)\n","Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from boto3<2.0,>=1.26.28->sagemaker) (0.10.0)\n","Requirement already satisfied: zipp>=0.5 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from importlib-metadata<5.0,>=1.4.0->sagemaker) (3.4.1)\n","Requirement already satisfied: typing-extensions>=3.6.4 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from importlib-metadata<5.0,>=1.4.0->sagemaker) (4.2.0)\n","Requirement already satisfied: pyparsing>=2.0.2 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from packaging>=20.0->sagemaker) (2.4.7)\n","Requirement already satisfied: six>=1.9 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from protobuf<4.0,>=3.1->sagemaker) (1.15.0)\n","Requirement already satisfied: python-dateutil>=2.7.3 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pandas->sagemaker) (2.8.1)\n","Requirement already satisfied: pytz>=2017.3 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pandas->sagemaker) (2021.1)\n","Requirement already satisfied: ppft>=1.7.6.6 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pathos->sagemaker) (1.7.6.6)\n","Requirement already satisfied: multiprocess>=0.70.14 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pathos->sagemaker) (0.70.14)\n","Requirement already satisfied: dill>=0.3.6 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pathos->sagemaker) (0.3.6)\n","Requirement already satisfied: pox>=0.3.2 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from pathos->sagemaker) (0.3.2)\n","Requirement already satisfied: contextlib2>=0.5.5 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from schema->sagemaker) (21.6.0)\n","Requirement already satisfied: urllib3<1.27,>=1.25.4 in c:\\users\\admin\\appdata\\local\\programs\\python\\python37\\lib\\site-packages (from botocore<1.30.0,>=1.29.47->boto3<2.0,>=1.26.28->sagemaker) (1.26.14)\n"]}],"source":["!pip install sagemaker"]},{"cell_type":"code","execution_count":4,"metadata":{"collapsed":true,"execution":{"iopub.execute_input":"2023-01-11T13:52:38.805903Z","iopub.status.busy":"2023-01-11T13:52:38.805496Z","iopub.status.idle":"2023-01-11T13:52:40.851601Z","shell.execute_reply":"2023-01-11T13:52:40.850132Z","shell.execute_reply.started":"2023-01-11T13:52:38.805869Z"},"jupyter":{"outputs_hidden":true},"trusted":true},"outputs":[],"source":["import sagemaker\n","import boto3"]},{"cell_type":"code","execution_count":5,"metadata":{},"outputs":[],"source":["from sagemaker.huggingface.model import HuggingFaceModel\n","\n","role = CFG[\"iam_role\"]\n","\n","hub = {\n","\t'HF_MODEL_ID': CFG[\"HF_MODEL_ID\"],\n","\t'HF_TASK':CFG[\"HF_TASK\"]\n","}\n","\n","# create Hugging Face Model Class\n","huggingface_model = HuggingFaceModel(\n","\ttransformers_version='4.17.0',\n","\tpytorch_version='1.10.2',\n","\tpy_version='py38',\n","\tenv=hub,\n","\trole=role, \n",")"]},{"cell_type":"code","execution_count":6,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["------!"]}],"source":["# deploy model to SageMaker Inference\n","predictor = huggingface_model.deploy(\n","\tinitial_instance_count=1, # number of instances\n","\tinstance_type='ml.m5.xlarge' # ec2 instance type\n",")"]},{"cell_type":"code","execution_count":7,"metadata":{},"outputs":[{"data":{"text/plain":["[{'generated_text': 'Swiss National Bank reports record loss for 2022 financial year'}]"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":["predictor.predict({\n","\t'inputs': \"The Swiss National Bank on Monday reported a loss of 132 billion Swiss francs ($143 billion) for the 2022 financial year, citing preliminary figures.It represents the biggest loss in the central bank’s 116-year history and equates to roughly 18% of Switzerland’s projected gross domestic product of 744.5 billion Swiss francs. Its previous record loss was 23 billion francs in 2015. As a result it will not make its usual payouts to the Swiss government and member states, it said, with payments to its shareholders also set to be affected. In 2021, the bank reported a 26 billion franc profit. Of the losses, 131 billion francs came from its foreign currency positions and 1 billion from its Swiss franc positions amid strong gains made by the franc as investors flocked to the perceived safe haven amid European volatility. Since June 2022, the Swiss franc has been trading above one euro, a level it had previously only briefly touched in 2015 after scrapping its 1.20 peg to the EU’s single currency. Switzerland has historically attempted to rein in the strength of the franc because of its export-heavy economy, though analysts have argued Swiss businesses have been able to remain competitive despite the rising franc due to euro zone inflation. In December, the Swiss National Bank raised interest rates for the third time in 2022, to 1%. That was to counter inflation of 3% — well below the euro zone’s inflation rate, which remains above 10%. The SNB was also impacted last year by losses in its stock and bond portfolio amid the wider market downturn. However, it gained 400 million francs through its gold holdings. Karsten Junius, chief economist at Swiss bank J.Safra Sarasin, told CNBC that the central bank’s losses would not alter its monetary policy and he expected another 100 basis points of hikes, to 2%, this year. “While the SNB will also need some time to rebuild its valuation reserves it will take less time to show profits than in the case of the European Central Bank,” he said, also noting that inflation in Switzerland was closer to its 2% target than in the euro zone. “While both central banks are structurally profitable as they can renumerate their liabilities at a lower rate than the market, the SNB will earn higher market interest this year already while the ECB is stuck with its low yielding bonds in its book and will be unprofitable for many years.”\"\n","})"]},{"attachments":{},"cell_type":"markdown","metadata":{},"source":["# Using Deployed Sagemaker Model"]},{"cell_type":"code","execution_count":68,"metadata":{},"outputs":[],"source":["client = boto3.client('sagemaker')\n","endpoints = client.list_endpoints()"]},{"cell_type":"code","execution_count":69,"metadata":{},"outputs":[],"source":["for e in endpoints[\"Endpoints\"]:\n","    if e[\"EndpointName\"] == CFG[\"endpoint_name\"]:\n","        name = e[\"EndpointName\"]\n","        mme_name = name"]},{"cell_type":"code","execution_count":70,"metadata":{},"outputs":[],"source":["sess=sagemaker.Session()\n","\n","predictor=sagemaker.predictor.Predictor(endpoint_name=mme_name, sagemaker_session=sess)\n","predictor.serializer = sagemaker.serializers.JSONSerializer()\n","predictor.deserializer = sagemaker.deserializers.JSONDeserializer()"]},{"cell_type":"code","execution_count":71,"metadata":{},"outputs":[],"source":["data = {\n","\t\"inputs\": \"The Swiss National Bank on Monday reported a loss of 132 billion Swiss francs ($143 billion) for the 2022 financial year, citing preliminary figures.It represents the biggest loss in the central bank’s 116-year history and equates to roughly 18% of Switzerland’s projected gross domestic product of 744.5 billion Swiss francs. Its previous record loss was 23 billion francs in 2015. As a result it will not make its usual payouts to the Swiss government and member states, it said, with payments to its shareholders also set to be affected. In 2021, the bank reported a 26 billion franc profit. Of the losses, 131 billion francs came from its foreign currency positions and 1 billion from its Swiss franc positions amid strong gains made by the franc as investors flocked to the perceived safe haven amid European volatility. Since June 2022, the Swiss franc has been trading above one euro, a level it had previously only briefly touched in 2015 after scrapping its 1.20 peg to the EU’s single currency. Switzerland has historically attempted to rein in the strength of the franc because of its export-heavy economy, though analysts have argued Swiss businesses have been able to remain competitive despite the rising franc due to euro zone inflation. In December, the Swiss National Bank raised interest rates for the third time in 2022, to 1%. That was to counter inflation of 3% — well below the euro zone’s inflation rate, which remains above 10%. The SNB was also impacted last year by losses in its stock and bond portfolio amid the wider market downturn. However, it gained 400 million francs through its gold holdings. Karsten Junius, chief economist at Swiss bank J.Safra Sarasin, told CNBC that the central bank’s losses would not alter its monetary policy and he expected another 100 basis points of hikes, to 2%, this year. “While the SNB will also need some time to rebuild its valuation reserves it will take less time to show profits than in the case of the European Central Bank,” he said, also noting that inflation in Switzerland was closer to its 2% target than in the euro zone. “While both central banks are structurally profitable as they can renumerate their liabilities at a lower rate than the market, the SNB will earn higher market interest this year already while the ECB is stuck with its low yielding bonds in its book and will be unprofitable for many years.” \",\n","\t'parameters': {\n","\t\t\"num_return_sequences\": 5\n","\t}\n","}"]},{"cell_type":"code","execution_count":72,"metadata":{},"outputs":[{"data":{"text/plain":["[{'generated_text': 'Swiss National Bank reports record loss for 2022 financial year'},\n"," {'generated_text': 'Swiss National Bank reports record loss of 132 billion francs in 2022'},\n"," {'generated_text': 'Swiss National Bank reports record loss of 132 billion francs for 2022'},\n"," {'generated_text': 'Swiss National Bank reports record loss of 132 billion Swiss francs in 2022'},\n"," {'generated_text': 'Swiss National Bank reports record loss of 132 billion francs for 2022 financial year'}]"]},"execution_count":72,"metadata":{},"output_type":"execute_result"}],"source":["predictor.predict(data)"]},{"attachments":{},"cell_type":"markdown","metadata":{},"source":["# Accessing Model through Hugging Face API"]},{"cell_type":"code","execution_count":133,"metadata":{},"outputs":[],"source":["import json\n","import requests\n","api_key = CFG[\"hf_apikey\"]\n","headers = {\"Authorization\": f\"Bearer {api_key}\"}\n","API_URL = CFG[\"hf_apiurl\"]\n","def query(payload):\n","    data = json.dumps(payload)\n","    response = requests.request(\"POST\", API_URL, headers=headers, data=data)\n","    return json.loads(response.content.decode(\"utf-8\"))\n","data = query(\n","    {\n","        \"inputs\": [\"TEST\", \"The Swiss National Bank on Monday reported a loss of 132 billion Swiss francs ($143 billion) for the 2022 financial year, citing preliminary figures.It represents the biggest loss in the central bank’s 116-year history and equates to roughly 18% of Switzerland’s projected gross domestic product of 744.5 billion Swiss francs. Its previous record loss was 23 billion francs in 2015. As a result it will not make its usual payouts to the Swiss government and member states, it said, with payments to its shareholders also set to be affected. In 2021, the bank reported a 26 billion franc profit. Of the losses, 131 billion francs came from its foreign currency positions and 1 billion from its Swiss franc positions amid strong gains made by the franc as investors flocked to the perceived safe haven amid European volatility. Since June 2022, the Swiss franc has been trading above one euro, a level it had previously only briefly touched in 2015 after scrapping its 1.20 peg to the EU’s single currency. Switzerland has historically attempted to rein in the strength of the franc because of its export-heavy economy, though analysts have argued Swiss businesses have been able to remain competitive despite the rising franc due to euro zone inflation. In December, the Swiss National Bank raised interest rates for the third time in 2022, to 1%. That was to counter inflation of 3% — well below the euro zone’s inflation rate, which remains above 10%. The SNB was also impacted last year by losses in its stock and bond portfolio amid the wider market downturn. However, it gained 400 million francs through its gold holdings. Karsten Junius, chief economist at Swiss bank J.Safra Sarasin, told CNBC that the central bank’s losses would not alter its monetary policy and he expected another 100 basis points of hikes, to 2%, this year. “While the SNB will also need some time to rebuild its valuation reserves it will take less time to show profits than in the case of the European Central Bank,” he said, also noting that inflation in Switzerland was closer to its 2% target than in the euro zone. “While both central banks are structurally profitable as they can renumerate their liabilities at a lower rate than the market, the SNB will earn higher market interest this year already while the ECB is stuck with its low yielding bonds in its book and will be unprofitable for many years.”\"],\n","        \"options\": {\"wait_for_model\":True},\n","        }\n","    )"]},{"cell_type":"code","execution_count":134,"metadata":{},"outputs":[{"data":{"text/plain":["[{'generated_text': 'TEST'},\n"," {'generated_text': 'The Swiss National Bank on Monday reported the biggest loss in its 116-year history and said it would not make its usual payouts to the Swiss government and member states.'}]"]},"execution_count":134,"metadata":{},"output_type":"execute_result"}],"source":["data"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit (AMD64)]"},"vscode":{"interpreter":{"hash":"252189e587d1e2aeba4a06e91fa71896c7a7f6e22e918b9407c7cde4ef2d5985"}}},"nbformat":4,"nbformat_minor":4}