Manage IBM Watson Studio packages as a user
You can add processing capabilities to a base image by installing custom, external, or third-party libraries and packages. The updated image can then be reused by other users.
IBM® Watson™ Studio includes many preinstalled libraries. Before you install a library, verify that it isn't preinstalled by running the appropriate command from a notebook cell:
- Python
-
!pip list --isolated
- R
-
installed.packages()
The format for library packages depends on the programming language.
Install a Python library for personal use
- Use the Python pip package installer command to install Python libraries to
your notebook. For example:
- To install the
prettyplotlib
library into a pod, run the following command:!pip install prettyplotlib
When the pod is terminated, the package is removed.
- To install the
prettyplotlib
in your user home directory (/user-home/user_name), run the following command:!pip install --user prettyplotlib
The
--user
flag installs the library for your personal use. The installed package can be used by all notebooks that use the same Python version in the Spark service. This package persists even if the pod is terminated.To use the package, you must restart your environment.
- To install the
- Use the Python import command to import the library components. For example,
run the following command in a code cell:
import prettyplotlib as ppl
- Restart the kernel.
Import custom functions defined in Python scripts for personal use
You can import a python script to the Scripts section of your project, and
then import the functions like any other Python module. The following example imports all functions
in a file named dsxl_support_functions.py
:
sys.path.insert(0, '../scripts/')
from dsxl_support_functions import *
In this example, ../scripts is the location where you imported your Python scripts. The imported functions can be called like any regular functions defined in modules.
Install libraries and packages in a custom Jupyter image for anyone to use
You can install a library or package in the root conda
environment. You can then
save the running environment as an image so that it can be shared with other users.
To import a package or library in a custom Jupyter image:
- Open a notebook or terminal in the running Jupyter environment. You can open a terminal from the Launch Terminal icon ( ).
- Install the library or package using
conda
, for example,conda install -y arrow
. - Go to Environments and click the Save icon () for the running environment to save it into a new custom image on the cluster.
- Name and tag the new custom image. Click the Save button.
To view or delete this new custom image, click the menu icon ( ) and click My Images.
Any user can select this new image by clicking Edit settings next to the corresponding environment.
An administrator can see, manage, or validate user-created images from the administration console. For more information, see Managing images .
Use a Scala library
Libraries for Scala notebooks are typically packaged as Java™ archive (JAR) files.
The libraries for a Scala notebook are not installed to the Spark service. Instead, the libraries are cached when they are downloaded and are only available for the time that the notebook runs.
- To use a single library without dependencies, from a public web server:
- Locate the publicly available URL to the library that you want to install. If you create a custom library, you can post it to any publicly available repository, such as GitHub.
- Download the library that you want to use in your notebook by running the following command in a
code cell:
%AddJar URL_to_jar_file
- To use a library with dependencies, from a public Maven repository:
- Add and import a library with all its dependencies by running the following command. You need
the groupId, artifactId, and version of the
dependency. For
example:
%AddDepsorg.apache.spark spark-streaming-kafka_2.10 1.1.0--transitive
- Add and import a library with all its dependencies by running the following command. You need
the groupId, artifactId, and version of the
dependency. For
example:
Load an R package
- Use the R install.packages() function to install new R packages. For example,
run the following command in a code cell to install the
ggplot2
package for plotting functions:install.packages("ggplot2")
The imported package can be used by all R notebooks running in the Spark service.
- Use the R library() function to load the installed package. For example, run
the following command in a code cell:
library("ggplot2")
You can now call plotting functions from the ggplot2 package in your notebook.