REST API example (Decision Optimization)
You can deploy a Decision Optimization model, create and monitor jobs and get solutions using the Watson Machine Learning REST API.
About this task
For Windows users, use ^ instead of \ for the multi-line separator and double quotation marks " throughout these code examples. Windows users also need to use indentation of at least one character space in the header lines.
For clarity, some code examples in this procedure have been placed in a json file to make the commands more readable and easier to use.
Once you have created a deployment using the REST API, you can also view it and send jobs to it from the Deployment spaces page in the user interface.
Procedure
- Authenticate as follows using your host cluster.
curl -X GET https://HOST-CLUSTER-HERE/v1/preauth/validateAuth -u USER-NAME-HERE:PASSWORD-HEREOutput example:{ "username":"USER-NAME", "role":"User", "permissions":["can_provision","manage_catalog","access_catalog"], "groups":[10000], "sub":"USER-NAME", "iss":"KNOXSSO", "aud":"DSX", "uid":"1000331010", "authenticator":"default", "accessToken":"************ obtained Token ********************", "messageCode":"success", "message":"success" } - (Optional) If you have not obtained your
SPACE-ID from the user interface as described previously, you can create a space using the
REST API as follows. Use the previously obtained IAM token prepended by the word
bearerin theAuthorizationheader in all API calls.curl -X POST \ https://HOST-CLUSTER-HERE/v2/spaces \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -d "{"name": "SPACE-NAME-HERE","description": "optional description here"}"For Windows users, put the--data-rawcommand on one line and replace all"with\"inside this command as follows:
Alternatively put the data in a separate file.curl --request POST ^ "https://HOST-CLUSTER-HERE/v2/spaces" ^ -H "Authorization: Bearer TOKEN-HERE" ^ -H "Content-Type: application/json" ^ --data-raw "{\"name\": \"SPACE-NAME-HERE\",\"description\": \"optional description here\"}"A SPACE-ID is returned inidfield of themetadatasection.Output example:
{ "entity": { "compute": [{ "crn": "crn:v1:cpd:private:pm-20:private:a/cpduser:99999999-9999-9999-9999-999999999999::", "guid": "99999999-9999-9999-9999-999999999999", "name": "Watson Machine Learning", "type": "machine_learning" }], "name": "SPACE-NAME", "scope": { "bss_account_id": "cpdaccount" }, "status": { "state": "active" } }, "metadata": { "created_at": "2020-10-30T16:16:44.121Z", "creator_id": "XXXXXXX", "id": "SPACE-ID", "url": "/v2/spaces/SPACE-ID" } }You must wait until your deployment space status is
"active"before continuing. You can poll to check for this as follows.curl --request GET "https://HOST-CLUSTER-HERE/v2/spaces/SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" - Create a new Decision Optimization model
All API requests require a version parameter that takes a date in the format
version=YYYY-MM-DD. This code example posts a model that uses the filecreate_model.json. The URL will vary according to the chosen region/location for your machine learning service.
The create_model.json file contains the following code:curl --request POST \ "https://HOST-CLUSTER-HERE/ml/v4/models?version=2020-08-01" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -d @create_model.json{ "name": "ModelName", "description": "ModelDescription", "type": "do-docplex_20.1", "software_spec": { "name": "do_20.1" }, "custom": { "decision_optimization": { "oaas.docplex.python": "3.9" } }, "space_id": "SPACE-ID-HERE" }The Python version is stated explicitly here in a
customblock. This is optional. Without it your model will use the default version which is currently Python 3.9. As the default version will evolve over time, stating the Python version explicitly enables you to easily change it later or to keep using an older supported version when the default version is updated. Currently supported versions are Python 3.9.If you want to be able to run jobs for this model from the user interface, instead of only using the REST API , you must define the schema for the input and output data. If you do not define the schema when you create the model, you can only run jobs using the REST API and not from the user interface.
Alternative (optional) code to define schema (enabling jobs to be run from the user interface):
{ "name": "Diet-Model-schema", "description": "Diet", "type": "do-docplex_20.1", "schemas": { "input": [ { "id": "diet_food_nutrients", "fields": [ { "name": "Food", "type": "string" }, { "name": "Calories", "type": "double" }, { "name": "Calcium", "type": "double" }, { "name": "Iron", "type": "double" }, { "name": "Vit_A", "type": "double" }, { "name": "Dietary_Fiber", "type": "double" }, { "name": "Carbohydrates", "type": "double" }, { "name": "Protein", "type": "double" } ] }, { "id": "diet_food", "fields": [ { "name": "name", "type": "string" }, { "name": "unit_cost", "type": "double" }, { "name": "qmin", "type": "double" }, { "name": "qmax", "type": "double" } ] }, { "id": "diet_nutrients", "fields": [ { "name": "name", "type": "string" }, { "name": "qmin", "type": "double" }, { "name": "qmax", "type": "double" } ] } ], "output": [ { "id": "solution", "fields": [ { "name": "name", "type": "string" }, { "name": "value", "type": "double" } ] } ] }, "software_spec": { "name": "do_20.1" }, "space_id": "SPACE-ID-HERE" }When you post a model you provide information about its model type and the software specification to be used.Model types can be, for example:do-opl_20.1for OPL modelsdo-cplex_20.1for CPLEX modelsdo-cpo_20.1for CP modelsdo-docplex_20.1for Python models
Versions 12.9 and 12.10 can also be used for these model types, but these are now deprecated and will soon be removed. Version 12.9 will be removed in the next version.
For the software specification, you can use the default specifications using their names
do_20.1ordo_12.10ordo_12.9(do_12.10 and 12.9 runtimes are deprecated, and will soon be removed). See also Extend software specification notebook which shows you how to extend the Decision Optimization software specification (runtimes with additional Python libraries for docplex models).Important: As CPLEX runtimes do_12.9 and do_12.10 will be removed in the next release of Cloud Pak for Data, start using do_20.1. Make sure that all your existing deployments use do_20.1 before upgrading to the next release. Otherwise any outdated deployments will be removed during the upgrade and you will no longer have access to them.To update an existing deployment to use a later software specification (CPLEX runtime), you can use either the REST API or the UI.
A MODEL-ID is returned in
idfield in themetadata.Output example:{ "entity": { "software_spec": { "id": "SOFTWARE-SPEC-ID" }, "type": "do-docplex_20.1" }, "metadata": { "created_at": "2020-07-17T08:37:22.992Z", "description": "ModelDescription", "id": "MODEL-ID", "modified_at": "2020-07-17T08:37:22.992Z", "name": "ModelName", "owner": "***********", "space_id": "SPACE-ID" }, "system": { "warnings": [] }} -
Upload a Decision Optimization model formulation
ready for deployment. First compress your model into a (
tar.gz or .zip) file and upload it to be deployed by the Watson Machine Learning service.This code example uploads a model called diet.zip that contains a Python model and no common data:curl --request PUT \ "https://HOST-CLUSTER-HERE/ml/v4/models/MODEL-ID-HERE/content?version=2020-08-01&space_id=SPACE-ID-HERE&content_format=native" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/gzip" \ --data-binary "@diet.zip"You can download this example and other models from the DO-samples. Select the relevant product and version subfolder. - Deploy your model Create a reference to your model. Use the SPACE-ID, the MODEL-ID obtained when you created your model ready for deployment and the hardware specification. For example:
curl --request POST "https://HOST-CLUSTER-HERE/ml/v4/deployments?version=2020-08-01" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -d @deploy_model.jsonThe deploy_model.json file contains the following code:{ "name": "Test-Diet-deploy", "space_id": "SPACE-ID-HERE", "asset": { "id": "MODEL-ID-HERE" }, "hardware_spec": { "name": "S" }, "batch": {} }The DEPLOYMENT-ID is returned inidfield in themetadata. Output example:{ "entity": { "asset": { "id": "MODEL-ID" }, "custom": {}, "description": "", "hardware_spec": { "id": "HARDWARE-SPEC-ID", "name": "S", "num_nodes": 1 }, "name": "Test-Diet-deploy", "space_id": "SPACE-ID", "status": { "state": "ready" } }, "metadata": { "created_at": "2020-07-17T09:10:50.661Z", "description": "", "id": "DEPLOYMENT-ID", "modified_at": "2020-07-17T09:10:50.661Z", "name": "test-Diet-deploy", "owner": "**************", "space_id": "SPACE-ID" } } -
Once deployed, you can monitor your model's deployment state. Use the
DEPLOYMENT-ID.
For example:
curl --request GET "https://HOST-CLUSTER-HERE/ml/v4/deployments/DEPLOYMENT-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json"Output example:
{ "entity": { "asset": { "id": "MODEL-ID" },"batch": { },"custom": {}, "description": "", "hardware_spec": { "id": "HARDWARE-SPEC-ID", "name": "S", "num_nodes": 1 }, "name": "Test-Diet-deploy", "space_id": "SPACE-ID", "status": { "state": "ready" } }, "metadata": { "created_at": "2020-11-04T16:22:14.442Z", "description": "", "id": "DEPLOYMENT-ID", "modified_at": "2020-11-04T16:22:14.442Z", "name": "test-Diet-deploy", "owner": "**************", "space_id": "SPACE-ID" } } - You can then Submit jobs for your deployed model defining the input data and the
output (results of the optimization solve) and the log file. For example, the following shows the contents of a file called
myjob.json. It contains (inline) input data, some solve parameters, and specifies that the output will be a .csv file. For examples of other types of input data references, see Model input and output data adaptation (Decision Optimization).
This code example posts a job that uses this file{ "name":"test-job-diet", "space_id": "SPACE-ID-HERE", "deployment": { "id": "DEPLOYMENT-ID-HERE" }, "decision_optimization" : { "solve_parameters" : { "oaas.logAttachmentName":"log.txt", "oaas.logTailEnabled":"true" }, "input_data": [ { "id":"diet_food.csv", "fields" : ["name","unit_cost","qmin","qmax"], "values" : [ ["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10] ] }, { "id":"diet_food_nutrients.csv", "fields" : ["Food","Calories","Calcium","Iron","Vit_A","Dietary_Fiber","Carbohydrates","Protein"], "values" : [ ["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4 ] ] }, { "id":"diet_nutrients.csv", "fields" : ["name","qmin","qmax"], "values" : [ ["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100] ] } ], "output_data": [ { "id":".*\\.csv" } ] } }myjob.json.curl --request POST "https://HOST-CLUSTER-HERE/ml/v4/deployment_jobs?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json" \ -H "cache-control: no-cache" \ -d @myjob.jsonA JOB-ID is returned. Output example: (the job is queued){ "entity": { "decision_optimization": { "input_data": [{ "id": "diet_food.csv", "fields": ["name", "unit_cost", "qmin", "qmax"], "values": [["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10]] }, { "id": "diet_food_nutrients.csv", "fields": ["Food", "Calories", "Calcium", "Iron", "Vit_A", "Dietary_Fiber", "Carbohydrates", "Protein"], "values": [["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4]] }, { "id": "diet_nutrients.csv", "fields": ["name", "qmin", "qmax"], "values": [["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100]] }], "output_data": [ { "id": ".*\\.csv" } ], "solve_parameters": { "oaas.logAttachmentName": "log.txt", "oaas.logTailEnabled": "true" }, "status": { "state": "queued" } }, "deployment": { "id": "DEPLOYMENT-ID" }, "platform_job": { "job_id": "", "run_id": "" } }, "metadata": { "created_at": "2020-07-17T10:42:42.783Z", "id": "JOB-ID", "name": "test-job-diet", "space_id": "SPACE-ID" } } - You can also monitor job states. Use the JOB-ID For example:
curl --request GET \ "https://HOST-CLUSTER-HERE/ml/v4/deployment_jobs/JOB-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE" \ -H "Content-Type: application/json"Output example: (job has completed){ "entity": { "decision_optimization": { "input_data": [{ "fields": ["name", "unit_cost", "qmin", "qmax"], "id": "diet_food.csv", "values": [["Roasted Chicken", 0.84, 0, 10], ["Spaghetti W/ Sauce", 0.78, 0, 10], ["Tomato,Red,Ripe,Raw", 0.27, 0, 10], ["Apple,Raw,W/Skin", 0.24, 0, 10], ["Grapes", 0.32, 0, 10], ["Chocolate Chip Cookies", 0.03, 0, 10], ["Lowfat Milk", 0.23, 0, 10], ["Raisin Brn", 0.34, 0, 10], ["Hotdog", 0.31, 0, 10]] }, { "fields": ["Food", "Calories", "Calcium", "Iron", "Vit_A", "Dietary_Fiber", "Carbohydrates", "Protein"], "id": "diet_food_nutrients.csv", "values": [["Spaghetti W/ Sauce", 358.2, 80.2, 2.3, 3055.2, 11.6, 58.3, 8.2], ["Roasted Chicken", 277.4, 21.9, 1.8, 77.4, 0, 0, 42.2], ["Tomato,Red,Ripe,Raw", 25.8, 6.2, 0.6, 766.3, 1.4, 5.7, 1], ["Apple,Raw,W/Skin", 81.4, 9.7, 0.2, 73.1, 3.7, 21, 0.3], ["Grapes", 15.1, 3.4, 0.1, 24, 0.2, 4.1, 0.2], ["Chocolate Chip Cookies", 78.1, 6.2, 0.4, 101.8, 0, 9.3, 0.9], ["Lowfat Milk", 121.2, 296.7, 0.1, 500.2, 0, 11.7, 8.1], ["Raisin Brn", 115.1, 12.9, 16.8, 1250.2, 4, 27.9, 4], ["Hotdog", 242.1, 23.5, 2.3, 0, 0, 18, 10.4]] }, { "fields": ["name", "qmin", "qmax"], "id": "diet_nutrients.csv", "values": [["Calories", 2000, 2500], ["Calcium", 800, 1600], ["Iron", 10, 30], ["Vit_A", 5000, 50000], ["Dietary_Fiber", 25, 100], ["Carbohydrates", 0, 300], ["Protein", 50, 100]] }], "output_data": [{ "fields": ["Name", "Value"], "id": "kpis.csv", "values": [["Total Calories", 2000], ["Total Calcium", 800.0000000000001], ["Total Iron", 11.278317739831891], ["Total Vit_A", 8518.432542485823], ["Total Dietary_Fiber", 25], ["Total Carbohydrates", 256.80576358904455], ["Total Protein", 51.17372234135308], ["Minimal cost", 2.690409171696264]] }, { "fields": ["name", "value"], "id": "solution.csv", "values": [["Spaghetti W/ Sauce", 2.1551724137931036], ["Chocolate Chip Cookies", 10], ["Lowfat Milk", 1.8311671008899097], ["Hotdog", 0.9296975991385925]] }], "output_data_references": [], "solve_parameters": { "oaas.logAttachmentName": "log.txt", "oaas.logTailEnabled": "true" }, "solve_state": { "details": { "KPI.Minimal cost": "2.690409171696264", "KPI.Total Calcium": "800.0000000000001", "KPI.Total Calories": "2000.0", "KPI.Total Carbohydrates": "256.80576358904455", "KPI.Total Dietary_Fiber": "25.0", "KPI.Total Iron": "11.278317739831891", "KPI.Total Protein": "51.17372234135308", "KPI.Total Vit_A": "8518.432542485823", "MODEL_DETAIL_BOOLEAN_VARS": "0", "MODEL_DETAIL_CONSTRAINTS": "7", "MODEL_DETAIL_CONTINUOUS_VARS": "9", "MODEL_DETAIL_INTEGER_VARS": "0", "MODEL_DETAIL_KPIS": "[\"Total Calories\", \"Total Calcium\", \"Total Iron\", \"Total Vit_A\", \"Total Dietary_Fiber\", \"Total Carbohydrates\", \"Total Protein\", \"Minimal cost\"]", "MODEL_DETAIL_NONZEROS": "57", "MODEL_DETAIL_OBJECTIVE_SENSE": "minimize", "MODEL_DETAIL_TYPE": "LP", "PROGRESS_CURRENT_OBJECTIVE": "2.6904091716962637", "STAT.cplex.modelType": "LP", "STAT.cplex.size.booleanVariables": "0", "STAT.cplex.size.constraints": "7", "STAT.cplex.size.continousVariables": "9", "STAT.cplex.size.integerVariables": "0", "STAT.cplex.size.linearConstraints": "7", "STAT.cplex.size.quadraticConstraints": "0", "STAT.cplex.size.variables": "9" }, "latest_engine_activity": ["[2020-11-04T16:34:51Z, INFO] Model: diet", "[2020-11-04T16:34:51Z, INFO] - number of variables: 9", "[2020-11-04T16:34:51Z, INFO] - binary=0, integer=0, continuous=9", "[2020-11-04T16:34:51Z, INFO] - number of constraints: 7", "[2020-11-04T16:34:51Z, INFO] - linear=7", "[2020-11-04T16:34:51Z, INFO] - parameters: defaults", "[2020-11-04T16:34:51Z, INFO] - objective: minimize", "[2020-11-04T16:34:51Z, INFO] - problem type is: LP", "[2020-11-04T16:34:51Z, INFO] Warning: Model: \"diet\" is not a MIP problem, progress listeners are disabled", "[2020-11-04T16:34:51Z, INFO] objective: 2.690", "[2020-11-04T16:34:51Z, INFO] \"Spaghetti W/ Sauce\"=2.155", "[2020-11-04T16:34:51Z, INFO] \"Chocolate Chip Cookies\"=10.000", "[2020-11-04T16:34:51Z, INFO] \"Lowfat Milk\"=1.831", "[2020-11-04T16:34:51Z, INFO] \"Hotdog\"=0.930", "[2020-11-04T16:34:51Z, INFO] solution.csv" ], "solve_status": "optimal_solution" }, "status": { "completed_at": "2020-11-04T16:34:51.796Z", "running_at": "2020-11-04T16:34:50.592Z", "state": "completed" } }, "deployment": { "id": "DEPLOYMENT ID" }, "platform_job": { "job_id": "PLATFORM JOB ID", "run_id": "PLATFORM RUN ID" } }, "metadata": { "created_at": "2020-11-04T16:34:30.603Z", "id": "JOB-ID", "modified_at": "2020-11-04T16:34:51.881Z", "name": "test-job-diet", "space_id": "SPACE-ID" } } - (Optional) You can delete jobs as follows:
curl --request DELETE "https://HOST-CLUSTER-HERE/ml/v4/deployment_jobs/JOB-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE&hard_delete=true" \ -H "Authorization: bearer TOKEN-HERE"If you delete a job using the API, it will still be displayed in the user interface. - (Optional) You can delete deployments as follows:
curl --request DELETE "https://HOST-CLUSTER-HERE/ml/v4/deployments/DEPLOYMENT-ID-HERE?version=2020-08-01&space_id=SPACE-ID-HERE" \ -H "Authorization: bearer TOKEN-HERE"If you delete a deployment that contains jobs using the API, the jobs will still be displayed in the deployment space in the user interface.
Results
Once your model has been deployed and job executed, the solution results are provided either inline or in the file and location that you specified, for example using an S3 reference. You can post new jobs using the deployment-ID without having to redeploy your model.