Setup iPython notebook server for remote access

Post date: Apr 6, 2014 6:56:26 AM

Why you want this?

Your server is powerful, and huge amount of data is stored on it. You want to run your iPython job remotely on the server via browser on your local machine (laptop).

There are two steps:

install python, ipython on the server
create ipython profile on the server
create certificate on your server
run ipython on the server end so that you can login from browser from your local machine

Step-by-step procedure

Prerequisites: Let's say I have installed python and ipython on the server already

First, create ipython profile using command $ ipython profile create nbserver.

kittipat@ua41f725ec477518a8aee:~$ pwd

/home/local/ANT/kittipat

kittipat@ua41f725ec477518a8aee:~$ ipython profile create nbserver

[ProfileCreate] Generating default config file: u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver/ipython_config.py'

[ProfileCreate] Generating default config file: u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver/ipython_qtconsole_config.py'

[ProfileCreate] Generating default config file: u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver/ipython_notebook_config.py'

[ProfileCreate] Generating default config file: u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver/ipython_nbconvert_config.py'

then you will see "ipython" in .config directory. In the profile dir, you will see the following files:

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$ ls -al

total 120

drwxr-xr-x 6 kittipat domain^users  4096 Apr  5 21:45 .

drwxr-xr-x 3 kittipat domain^users  4096 Apr  5 21:45 ..

-rw-r--r-- 1 kittipat domain^users 18985 Apr  5 21:45 ipython_config.py

-rw-r--r-- 1 kittipat domain^users 27672 Apr  5 21:45 ipython_nbconvert_config.py

-rw-r--r-- 1 kittipat domain^users 22872 Apr  5 21:45 ipython_notebook_config.py

-rw-r--r-- 1 kittipat domain^users 23748 Apr  5 21:45 ipython_qtconsole_config.py

drwxr-xr-x 2 kittipat domain^users  4096 Apr  5 21:45 log

drwx------ 2 kittipat domain^users  4096 Apr  5 21:45 pid

drwx------ 2 kittipat domain^users  4096 Apr  5 21:45 security

drwxr-xr-x 2 kittipat domain^users  4096 Apr  5 21:45 startup

We will need to configure the config file ipython_config.py using vi:

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$ vi ipython_config.py

# Configuration file for ipython.

c = get_config()

# ================================================

#  Add your config here

# ================================================

# Kernel config

c.IPKernelApp.pylab = 'inline'  # if you want plotting support always

# Notebook config

c.NotebookApp.certfile = u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver/mycert.pem'

c.NotebookApp.ip = '*'

c.NotebookApp.open_browser = False

c.NotebookApp.password = u'sha1:650b271a3975:34dab072f6a38bab9272d9bb709bb4437857d438'

# It is a good idea to put it on a known, fixed port

c.NotebookApp.port = 9999

# ================================================

#------------------------------------------------------------------------------

# InteractiveShellApp configuration

#------------------------------------------------------------------------------

# A Mixin for applications that start InteractiveShell instances.

# Provides configurables for loading extensions and executing files as part of

# configuring a Shell environment.

# The following methods should be called by the :meth:`initialize` method of the

# subclass:

Note: The first part is the path to your (self-signed) certificate file (/home/local/ANT/kittipat/.config/ipython/profile_nbserver/mycert.pem). Please refer to "Appendix A: Create a certificate file" on how to create a certificate file. The second highlighted text is encrypted password, explained in "Appendix B: Create encrypted password".

Save the config file and now you are ready to use ipython remote console.

How to run it?

Go to any directory you want to run ipython on. Let's say "/home/local/ANT/kittipat/research/test_ipython/tutorial_ml_gkbionics-master"

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$ cd /home/local/ANT/kittipat/research/test_ipython/tutorial_ml_gkbionics-master

kittipat@ua41f725ec477518a8aee:~/research/test_ipython/tutorial_ml_gkbionics-master$ ipython notebook --profile=nbserver

2014-04-05 23:44:16.746 [NotebookApp] Using existing profile dir: u'/home/local/ANT/kittipat/.config/ipython/profile_nbserver'

2014-04-05 23:44:16.751 [NotebookApp] Using MathJax from CDN: https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js

2014-04-05 23:44:16.756 [NotebookApp] Serving notebooks from local directory: /home/local/ANT/kittipat/research/test_ipython/tutorial_ml_gkbionics-master

2014-04-05 23:44:16.756 [NotebookApp] The IPython Notebook is running at: https://[all ip addresses on your system]:9999/

2014-04-05 23:44:16.757 [NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

The response says that the profile directory is: /home/local/ANT/kittipat/.config/ipython/profile_nbserver

, and it runs from from: /home/local/ANT/kittipat/research/test_ipython/tutorial_ml_gkbionics-master

You can connect to the IPython Notebook session by https://[all ip addresses on your system]:9999/ from your local machine.

Now, you come back to your local computer (i.e., your laptop or desktop), and connect to the server. In my case my server is ua41f725ec477518a8aee.ant.amazon.com and I set the port to 9999 (default by IPython anyway), so the URL I will use is

https://ua41f725ec477518a8aee.ant.amazon.com:9999/

You will see something like this, just say "Proceed Anyway".

Then just login using your password you used in Appendix B.

If success, you will see notebooks in the dir /home/local/ANT/kittipat/research/test_ipython/tutorial_ml_gkbionics-master.

That's all. ;-)

Credits

All credits go to the following nice resources:

Running a notebook server

Access IPython remotely

Create an iPython HTML Notebook on Amazon's AWS Free Tier from scratch

Install IPython on AZURE.

Using IPython notebook on AZURE

Appendix A: Create a certificate file

Personally, I think it's quite convenient to have the certificate file on the same location as ipython config file, so I run the certificate creation command on the directory:

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

Generating a 1024 bit RSA private key

.....++++++

..................++++++

writing new private key to 'mycert.pem'

-----

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:US

State or Province Name (full name) [Some-State]:WA

Locality Name (eg, city) []:seattle

Organization Name (eg, company) [Internet Widgits Pty Ltd]:Botzilla

Organizational Unit Name (eg, section) []:gobbledygook

Common Name (e.g. server FQDN or YOUR name) []:youguess

Email Address []:me@botzilla.com

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$

From the ipython config file, you can just point to the certificate file location: /home/local/ANT/kittipat/.config/ipython/profile_nbserver/mycert.pem.

Appendix B: Created encrypted password

You can run this small python script, and input your password. This is the password to protect your notebook, so it has nothing to do with your login/password to the server (i.e. server and ipython notebook password can be different.)

kittipat@ua41f725ec477518a8aee:~/.config/ipython/profile_nbserver$ python -c "import IPython;print IPython.lib.passwd()"

Enter password:

Verify password:

sha1:650b271a3975:34dab072f6a38bab9272d9bb709bb4437857d438

You can just copy and paste the highlighted encrypted password into the specified area in the config file.