Daan van Eijk (dvaneijk@icecube.wisc.edu) and Nahee Park (npark@icecube.wisc.edu) edited these tutorials for the 2018 IceCube Bootcamp. The original versions were written by Kyle Jero and James Bourbeau for previous IceCube Bootcamps. The section 'Introduction to Python' was heavily influenced by two sources:
[ back to top ]
To follow these tutorials, you need to have a working installation of
In case you have any questions on installation of these packages or otherwise, please email Daan (daan.vaneijk@icecube.wisc.edu) or Nahee (npark@icecube.wisc.edu).
All the Python commands you can find in this tutorial can be run through one of the following ways:
[ back to top ]
More then likely you are sitting in front of a computer and reading this, if I told you to communicate with your computer and make it add some number or print out the current time what would you do? Odds are you would not start typing in the binary code so the computer would understand directly to make those things from scratch. As you will see soon you don't have to do that to get your point across effectively, though.
One picture you can keep in your head when imagining how programming works, at a very superficial level, is three people writing emails back and forth to each other. The first person is you, the second person is the interpreter, and the third person is your computer. You know how to write emails in English and soon python, the interpreter knows how to speak in python and the computer's language, and your computer knows only how to talk in its own language but can speak any language if given very explicit direction on how to do so.
So when you receive a task that you want to program, you translate English into python and write it down in an email to the interpreter. The interpreter receives this email and begins translating it into the computer's language, checking for errors as it goes. If an error is found, the interpreter will email you back and attempt to point out where your mistake was and what kind of error it thinks you made. Once the interpreter completes the entire translation and does not find any errors it will send that interpretation to your computer. When your computer receives the email it begins running the program described. The program may still have errors that the interpreter did not catch and if your computer catches them it will send an email to you specifying the error and it's location so you can email the interpreter with a modified version of the program. If there are no problems then the computer will complete any tasks you requested and write files to the system or email you back with things you requested from the results of the program.
Notice that at no point did the interpreter or your computer attempt to guess what you were trying to say, they simply read the emails and translate or run them. This means that if you write a program that is grammatically correct and can be run, it will run and neither the interpreter or your computer will warn you that things are wrong. One way to help prevent mistakes from happening is to ask your computer to be very verbose in it's replies the first few times it runs a program (print a lot of information) and verify that what its saying matches up with what your original idea was.
— Kyle Jero, wise hermit
[ back to top ]
On most Linux systems, bash is the default shell program. However, there are other shell programs that can be used as well (e.g. ksh, zsh, etc.).
Please open a terminal and try the following commands and tips on the command line:
This list of programs, their functions, and redirects/pipes is by no means complete, search the internet or ask others about commands to fulfill needs you have.
[ back to top ]
According to the official website, "The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more."
When properly installed, the Jupyter notebook app can be launched directly from the command line by typing:
jupyter notebook
The notebook operates in a similar way to the ipython terminal in that it evaluates cells to create programs. The difference is that in the notebook the cells can be more than one line long and you don't have to evaluate the cell to get a new one to write in. Additionally the notebook provides the ability to produce in-line graphics, use magic (see next section) and make comments in special cells that understand markdown and latex, making them a very powerful tool for documenting scientific programming. As an example of their usefulness these tutorials were also made in a Jupyter notebook. There are some drawbacks to this way of coding though, so don't forget to use the terminal and scripts when it's appropriate!
[ back to top ]
Magic is real, at least in Ipython and Jupyter notebooks. Magic commands start with a % and preform tasks outside of the scope of programming languages. For instance, at a certain you may find yourself thinking: wow, how am I going to remember all these commands I just entered in? Answer: with a little help from magic.
%history 1-10
The %history magic will print all the commands in the specified range to the screen. To save this output to a file we should use %save with the file to save to and the line numbers we want to save as arguments.
%save Python_Intro_History.txt 1-10
To see the syntax of a magic command you should use %command? where command is the magic you are interested in. Check out more magic commands on this Ipython quick reference.
[ back to top ]
The process of getting information from and putting it into a file is very important. What we'll cover here should only be used if you are parsing plain text, not a csv or other specialized format. There are much better ways to access formatted data and we will touch on those as they come up. But as an example, let's open a file, print out the contents line by line and then add text to the file.
We'll be working on the file named people.txt that you created in the section on the command line shell. If you didn't do that exercise, you can also download it here. Then, using your favorite text editor, open an empty file and save it as simplefilereadwrite.py. This will be your first Python script!
To open a file for writing use the built-in open() function. open() returns a file object, and is most commonly used with two arguments. The syntax is:
file_object = open(filename, mode)
where file_object is the Python variable to put the file object. The second argument describes the way in which the file will be used. The mode argument is optional; 'r' will be assumed if it’s omitted.
'r' when the file will only be read
'w' for only writing (an existing file with the same name will be erased)
'a' opens the file for appending; any data written to the file is automatically added to the end.
'r+' opens the file for both reading and writing.
Let's start typing in the Python script simplefilereadwrite.py (more information on writing Python scripts will follow later in the tutorials):
#!/usr/bin/env python
f=open("people.txt",'r+')
print(f.read())
The location of people.txt may be different for you, if it is not in your current directory you should give the path to it instead of just the file name. The f.read() command returns a string containing all characters in the file. Run this little script by entering on the command line:
python simplefilereadwrite.py
This should result in the same output as shown above. However, we said we wanted to do this line by line so we actually need a different way of doing this. When the file object is created by python the contents line by line can be iterated over with a for loop (more information on for loops will follow later in the tutorials). Thus we can adapt the script simplefilereadwrite.py to the following.
#!/usr/bin/env python
f=open("people.txt",'r+')
for line in f:
print(line)
The extra return between each line occurs because the print function and the file both produce a return giving us two. There are a number of ways to get rid of this effect, but I know that the hidden characters in the text file making this happen is the last one so we can get the results we want by just taking all the characters but the last one, recall that strings are actually lists (more information on lists will follow later in the tutorials).
#!/usr/bin/env python
f=open("people.txt",'r+')
for line in f:
print(line[:-1])
We also want to append a line to the end of the file so let's learn how to do this and then add in the argparser stuff. The write method takes one parameter, which is the string to be written. To start a new line after writing the data, add a \n character to the end. When we are done with a file we should close it.
f.write("Kevin 21\n")
f.close()