Steps

Alternatives

• You could search Google, YouTube, and many other places and find a billion hits
• You can get swamped with options
• This article is Nigel's starter pack for  a quick start.
• This information is what I found useful - Your needs can be different to mine!

What is Python good for?

• Data Scientist job & serious mega-bucks - You can double your already large salary!
• New new technology areas line PowerAI, Artificial Intelligence , Machine Learning, Deep Learning, and so on.
• Data manipulation to correct a files format and restructuring the data
• Web 2.0 web pages + REST API

How to develop code & run Python

1. Edit file and run file
2. IDE (integrated development environment)

• Initially IDE is a pain in the backside
  • As you have to learn both the IDE and Language together
  • Learning a IDE can take a month!
  • But good for a full time developer
• I recommend: edit and run but also you can run the python in console mode to try out things.
• I have written Python code for about a year. I think I am ready to try a IDE to edit code and debugging.
  • Probably the PyCharm Community Edition IDE for a start.

Environments

• Windows = yuck!
• Tablet - you can run PyCharms IDE but get yourself a Keyboard for typing.
• OSX = if you really have too!  Sorry never really got on with the Mac
• Linux = is the natural home of Python.  Although, AIX 7.3 comes with Python.
  • I am using a 160 CPU, 256 GB RAM, POWER8 S922LC - rather overkill but it is fast :-)
  • I also use a Raspberry Pi. For data files are not more than a GB it is quick. The Raspberry Pi memory is limited.
• AIX
  • it is in the AIX Open Source toolbox for downloading
  • take care with exotic modules as might have to use git & compile them yourself

How does Python run?

• Compiled - No like say C
• Interpreted - Yes but highly optimised, cached and parallelised.  I have some Python code that finishes so fast I assumed it crashed but it does work.

Which Python version 2.7 or 3.x ?

• 3.<latest> - at the rime of writing 3.5 to 3.7 depending on how current your OS is!
• No one is writing 2.7 any more
• But there is lots of it in use today but declining over time
• Not a massive difference but best to learn Python 3

Quick Assumption: You have, in the past, written some of these languages?

• C, Korn or bash shell script writing: excellent
• C programming: brilliant
• JavaScript programs: good
• Python Programming:  Why are you reading this article???

Then, you have already performance the heavy lifting.

Everyone can write a simple program!

  A=42  print "The number is " $A    if [[ $A == 42 ]]  then          print "eureka"  fi

Plus For loop & Functions

What is this? Well is work on my Korn Shell OK on AIX.

Mega Tip 1: If you know any of the languages listed, then Python is going to be simple

•   Updated  Read the magic Python by Example or watch the video by the originator Derek Banas
  • A big thanks for GjB in the comments for finding the link. I lost the link.
  • YouTube link (which is includes a link to the code examples): 
    • https://www.youtube.com/watch?v=N4mEzFDjqtA&t=3s
  • Approaching 5 million YouTube views - amazing for a techie video
  • The video is 43 minutes and fast paced.  A whole language with examples in under an hour.
  • The web page directly: 
    • http://www.newthinktank.com/2014/11/python-programming/
  • Full marks to Derek Banas the content.
• Briefly the contents

1. Data types:

• string,
• integers & float,
• tuples,
• lists,
• dictionary

1. Converting between them
2. Conditionals:  if, then, else
3. Loops:  for, while
4. Functions
5. User input
6. String manipulation
7. File I/O: read and write
8. Classes and objects
9. Inheritance            <-- IMHO advanced and for class module developers
10. Polymorphism       <-- IMHO advanced and for class module developers

Mega Tip 2: Socratica videos on YouTube

I have looks at many training course, Online content and YouTube series' and  Socratica videos are by far the best and free.

• Python Programming Tutorials (Computer Science)

• Concise with dry humour and some computer jokes - see recursion

• Mostly with worked example

• Excellent style

• Caltech grads

• 33 videos (Don't watch the two or three for Python2)

• Most ~8 minutes

• Total 3.5 hours
• 15 million views
• YouTube Socratica Playlist Videos
• A Geek person told me Socraticia is the female form of Socrates - I think the creators are female. They also cover advanced maths and SQL topics.
• 
• I watched all of these videos twice - about 6 months apart
• They are short but to consolidate what your learn try to have a quick go yourself on each topic

Mega Tip 3:  python.org = This website is the Python Mother Ship!

• https://python.org
• Then, Download the Alternative Platforms

• For the official documentation go to https://docs.python.org/3/tutorial/
• Tiny example 
• 

• Also, if you are stuck for the syntax of a statement or the details of some module or function then use then Google: python3 <your questions spelt out in full>
• Often you get http://Python3.org but http://stackoverflow.com answer with worked examples is good but scan down the answer a bit (the first might not be the best answer or exactly what you want)

Mega Tip 4: Get yourself a project to force you to code and work though problems and new features

• Something simple
• Something you are interested in
• Specially web focused
• Python strong at
  • Website interaction
  • REST API to an online service
  • Data manipulation and transformation
  • File conversion or filtering

Mega Idiot: My first project was the REST API to a HMC to extract Temp, Watts + performance stats for SSP, server & VM

• It was a BIG mistake
• The bad news was the API was so badly documented it was impossible to use!
• With complicated XML - using features that are rarely used by anyone.
• I had to interview the developers in the end to workout the hidden details of the REST API
• In simple terms, it was the "REST API from Hell!"
• But I learnt a lot

• In the end, I wrote a Python class module to hid the horrible REST API from Python programmers - its 1100 line of code.
• It returns simple to use Python data structures
• So in simply  ~40 lines of Python to extract, manipulate & save in:
  • CSV file,
  • .html with GoogleChart graphs
  • Insert into an influxDB database

Mega Tip 5: JSON format files are the same as the Python native data type called Dictionaries

• When learning Python, concentrate on Dictionaries
• Dictionaries are (simple)   { "some label": data, more here }
• and the data can be
  • "Strings" in double or single quote
  • Integers like 12345 or -42
  • Floating point numbers 123.456 (note the decimal point)
• Often we have a list of dictionaries - lists look like [ item, item, item, . . . ]

JSON file example of stats called "mydata.json":

  # 1st sample = Python dictionary  

{ "datetime": "2018-04-16T00:06:32",  
"cpus_active": 32,  
"mhz": 3521,  
"cpus_description": 
"PowerPC_POWER9.,  
"cpu_util": {            
    "user": 50.4,            
    "sys": 9.0,            
    "idle": 40.4,            
    "wait": 0.2
  }  
"disks": [
    { "name": "hdisk0", "size": 32, "busy": 12, "IOPS": 442 },    
    { "name": "hdisk1", "size": 128, "busy": 44, "IOPS": 1587 }
  ]   
}

Python Program to load the JSON data file -  NEW  Fixed a few Typos here, due to Cut'n'paste issues that is double quotes became full stops.

  # Read the file as plain text    
f = open("mydata.json","r")  
text = f.read()  f.close() 
   
# convert to Dictionary  
import json

#module to handle JSON format  
jdata = json.loads(text)

• That json.loads() function converts a string (text) to the dictionary called jdata at 10's of MBs of JSON a second.
• Now lets extract certain fields that use a natural Python syntax

  # get the Mhz from the record    
print("MHz=%d"%(jdata["mhz"]))    
# Loop through all the records pulling out the MHz numbers and the CPU utilisation user mode percent (its in sub dictionary called cpu_util)    

for sample in jdata["disks"]:      
  print("Size=%d"%(sample["size"]))

My latest project that uses Python is njmon for AIX and Linux - the new turbo nmon. 

• The J is for JSON and we use Python to make data handling easy.

• For AIX uses libperfstat C library - if you want details see: man libperfstat on AIX or vi /usr/include/libperfstat.h
  • Or find the worked example C code in KnowledgeCenter
• Status quirky but usable for expert C programmer
• Vast quantity of performance statistics (like 1000 for AIX and VIOS). If you have many disks, networks or ask for processes statistics then that grows rapidly.
• And for a bonus libperfstat gives us the current CPU MHz
• Similar for Linux
• njmon written in C to use C function into the UNIX kernel generates JSON data. Then, we use Python to accept the data and inject it live in to a Time Series Data fro graphing in real-time

Stand by for something strange

• "Well known" programming problem = swamping the values of two variables a and b. Classic solution is that use a temporary variable.

  temp = a
a = b
b = temp

• But can you do that without the temp variable?
• Not possible in C - I have known this for 40 years!
• Python answer

  a,b = b,a

• It is using a native data structure called a tuple.  As its a common programming task they built it into the language.
• Warning weirdness next:

• How about this?

  a = a + b
b = a - b
a = a - b

•   Wow! I thought it was impossible!

Next a tiny Web grabbing Python example

• Lots of websites and web services keep stats that you can download with your browser.
• I have used sourcfogre.net (used below) and youtube.com for examples.
• They are most often in JSON and Python has a requests module that makes "talking" to website simple
• As an example bung this in your browser ( NOT Internet Explorer )
  • https://sourceforge.net/projects/nmon/files/stats/json?start_date=2000-10-29&end_date=2020-12-31&os_by_country=false
• And you should get a load of JSON date back that Firefox and Chrome will organise and make pretty.

• Using Python, requests module and one of my own for graphing we can draw t downloads from the nmon project on SourceForge over time

• We also need to change the date format from which shows of some of Pythons simple data manipulation
• ,[2018-09-17 00:00:00],2
• to
• ,['Date(2018,9,17,00,00,00)', 2]
• Below is the source code - with many extra print lines and comments so if you run it would will see the data structures.
•  NEW  Changes the code here to NOT relying on my  nchart Python Module
• Green bits a debug but useful it you run it to see the data
• Red bits are the web-page pre-able and post-amble to setup the Googlechart library graph.

•   #!/usr/bin/python3  
  #--------------------------------- Get the data using REST API from sourceforge.net  
  import requests  
  URL='https://sourceforge.net/projects/nmon/files/stats/json?start_date=2000-10-29&end_date=2020-12-04&os_by_country=false'  
  
  ret = requests.get(URL)  
  print(ret.status_code)  
  #print("return code was %d"%(ret.status_code))  
  #print("characters returned %d"%(len(ret.text)))  
  
  #---------------------------------- Create dictionay  
  import json  
  jdata = json.loads(ret.text)  
  #print(jdata)  
  months=0  
  count=0  
  for row in jdata['downloads']:  
  #    print(row)      
    months=months+1      
    count=count+row[1]  
  print("months=%d"%(months))  
  print("count =%d"%(count))  
  
  #---------------------------------- Create web page+graph using Googlechart library  
  file = open("downloads.html","w")  
  file.write('<html>\n'  '  <head>\n'  
  '    <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>\n'  
  '    <script type="text/javascript">\n'  
  '      google.charts.load("current", {"packages":["corechart"]});\n'  
  '      google.charts.setOnLoadCallback(drawChart);\n'  
  '      function drawChart() {\n'  
  '        var data = google.visualization.arrayToDataTable([\n'  
  '[{type: "datetime", label: "Date"},"Files"]\n' )    
  
  for row in jdata['downloads']:      
    str=row[0]      
    str = str.replace("-",",")      
    str = str.replace(" ",",")      
    str = str.replace(":",",")      
    file.write(",['Date(%s)',%d]\n"%(str,row[1])) 
     
  file.write('        ]);\n'  
  '  var options = {title: "nmon Downloads", vAxis: {minValue: 0}};\n'
  '        var chart = new google.visualization.AreaChart(document.getElementById("chart_div"));\n'
  '        chart.draw(data, options);\n'  '      }\n'  '    </script>\n'  '  </head>\n'  
  '  <body>\n'  
  '    <div id="chart_div" style="width: 100%; height: 500px;"></div>\n'  
  '  </body>\n'  '</html>\n')  
  
  file.close()

• The output - skipping the dump of the JSON and the 105 rows of monthly stats looks like this

  ['2018-05-01 00:00:00', 14153]  
['2018-06-01 00:00:00', 12794]  
['2018-07-01 00:00:00', 12422]  
['2018-08-01 00:00:00', 13127]  
['2018-09-01 00:00:00', 11872]  
['2018-10-01 00:00:00', 13628]  
['2018-11-01 00:00:00', 12805]  
['2018-12-01 00:00:00', 15611]  
months=114  count =686634

• So that was captured in Jan  2019 and so far 686,634 downloads of nmon and its tools and the monthly download generated graph.

•  NEW  The generated downloads.html file has the following contents - note I removed a few 100 lines of data in the middle. Colours are from the vim editor - see later comments.
• So that was captured in Jan  2019 and so far 686,634 downloads of nmon and its tools and the monthly download generated graph looks like this:
•  NEW  Simpler graph
• 

C Programmers be aware:

I keep making the same mistakes in writing Python.

1. On Linux with the right export TERM=linux setting and using vi (actually vim) then you have syntax highlighting which reduces errors a lot - go for a white background or comments in dark blue are unreadable. See the picture below - I have not done that colouring - it is all vim.
2. vim also helps with auto indentation.
3. If, for and while statements have a ":" at the end of the line.
4. In Python it is print and in C it is printf - I had to teach my fingers to miss out the final "f"
5. Those maddening 4 stop indentations have to be exactly right!
6. Anything I missed?