.. _unix: ==================================== UNIX/LINUX operating system ==================================== Shells ++++++ A shell is a program that allows you to interact with the operating system and sofftwares through typing commands. As opposed to a graphical user interface (or GUI), which involves point-and-click, a shell is a command line interface (or CLI), where the user types in commands to interact with the computer. On a Mac machine there is the option of using a Unix shell in a "terminal window" (go to Applications –> Untilities –> Terminal to open a terminal window). The Mac operating system (macOS) is a Unix-based graphical operating system. The latest version of macOS is Catalina. A few previous versions include Mojave, Sierra, Leopard, Lion, Yosemite, and El Capitan. Two common Unix shells are `bash`__ and `csh`__. In this class you will mainly use the `bash` shell. In the shell (when you open a terminal window) you will have a ``prompt`` usually denoted by a single ``$`` where you can type in commands. If you think your shell is different, you can type: .. code-block:: none $ bash This will start a new bash shell and give you the bash prompt to type in commands. For example, type the command ``whoami`` at the prompt and hit return (or enter) to see what response you get. I get the following response: .. code-block:: none $ whoami motamed Now that our computer knows who we are, let's see what else the computer can do. **Remark**: Two other shells that you may already be familiar with include Matlab and Python. Most of you are already familiar with the Matlab shell (or "Matlab command window" if you are using a GUI version of Matlab). A Matlab shell uses the prompt ``>>``, instead of ``$`` used by bash. A Python sheel uses the prompt ``>>>``. To invoke and use Pyhton, simply type ``python``: .. code-block:: none $ python Python 2.7.13 (default, Apr 23 2017, 16:50:35) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> x=4 >>> y=2*x >>> y >>> 8 Type ``exit()`` to exit python and return to the shell. pwd, cd, and ls +++++++++++++++++++++++++++++++ The command ``pwd`` means "prints the working directory". The result will be the full path to where your prompt is located in the directory structure. Directories are called “folders” on windows systems. Another useful command is ``cd`` that is used to change working directories. For example if I am in the directory ``/home/motamed/`` and want to change to ``/home/motamed/TEACHING/MATH471``, where my files for the course Math 471 are stored, I would just type ``$ cd TEACHING/MATH471``, or alternatively ``$ cd TEACHING`` and then ``$ cd MATH471`` to go two levels down, one level at a time. If you want to go up (towards the top levels), you can use ``$ cd ..`` which brings you up one level. If you need to make a new directory use ``$ mkdir dir_name``, where ``dir_name`` is the name of the new directory chosen by you. The command ``ls`` lists the contents (i.e. files and directories) of the current working directory. It can be executed with different "options" (or flags); see ``man ls``. For example, the ``-lt`` option provides a long listing sorted by time modified. For this you just need to type: .. code-block:: none $ ls -lt The ``-l`` option tells ``ls`` to display a long listing that contains additional information about each file, such as how large it is, who owns it, when it was last modified, etc. The first 10 characters indicate the file type and read, write, and executing permission classes. See `here`__ for more information. Another example is the ``-a`` option, which displays also the hidden files, such as those started with a period (or dot). The command ``ls`` also takes "arguments", that is, a list of files to apply ``ls`` to. For example, if we want to list the information about a specific file (e.g. `git.rst` in my directory), we type: .. code-block:: none $ ls -l git.rst We can also use the ``*`` character to list more than one file: .. code-block:: none $ ls -l *.rst Displaying the contents of a file +++++++++++++++++++++++++++++++++++++++++++ The commands ``less``, ``more``, ``head``, ``tail``, and ``cat`` can be used to display the contents of files. The command ``less`` is somewhat more than ``more``, as it can scroll both up and down. The commands ``head`` and ``tail`` display the first and last n lines of a file, respectively. The default number of lines is n=10, but it can be adjusted to, say, 15 by typing ``$ tail -15 filename``. The command ``cat`` is also used to read files: ``$ cat filename``. We can also use ``cat`` to put its output (here "filename") to a new file or to over-write an existing file: ``$ cat filename > newfile`` or to append to an existing file ``$ cat filename >> existingfile``. Copying, moving, and deleting files +++++++++++++++++++++++++++++++++++++++++++ The commands ``cp``, ``mv``, and ``rm`` are used to copy, move, and delete files and directories, respectively. To copy a file named `original_file`, you can type: .. code-block:: none $ cp original_file copy_file To copy an entire directory structure named `original_dir` (i.e. copying all files and subdirectories in it), type: .. code-block:: none $ cp -r original_dir copy_dir To rename a file or move to a different place (e.g. a different directory), you can use the command ``mv``. You can for example type: .. code-block:: none $ mv oldfile newfile If you want to delete a file named `filename`, you can type: .. code-block:: none $ rm -i filename remove filename? The ``-i`` flag forces ``rm`` to ask before removing the file. Other useful flags for ``rm`` include ``-f`` for force to remove and ``-r`` for descending into sub-directories. Environment variables +++++++++++++++++++++ The bash shell uses environment variables to keep track of various information. The command ``printenv`` will display any environment variables you have set. Typing this command will give an output that looks something like this: .. code-block:: none [appelo@mizar ~]$ printenv HOSTNAME=mizar.unm.edu SHELL=/bin/bash USER=motamed PATH=/Users/motamed/mpich/bin:/usr/sbin:/sbin:/usr/local/bin:/sw/bin:/usr/bin PWD=/Users/motamed/TEACHING/MATH471/webpage HOME=/Users/motamed LOGNAME=motamed The environment variables can be accessed through the dollar sign, for example ``$ ls $PWD`` would list the content of the current working directory. OR ``$ cd $HOME/TEACHING/MATH579`` would take me to the subdirectory of the course Math 579 no matter where your current working directory is. You can print just one variable for example by: .. code-block:: none $ printenv PWD /Users/motamed/TEACHING/MATH471/webpage or .. code-block:: none $ echo $PWD /Users/motamed/TEACHING/MATH471/webpage You can also define new environment variables. For example, I can generate a new variable `HPSC` and define it to be the path to my course webpage directory: .. code-block:: none $ export HPSC=$HOME/TEACHING/MATH471/webpage Now try ``$ echo $HPSC``. PATH and the search path ++++++++++++++++++++++++++++ Whenever you type a command at the Unix prompt, the shell looks for a program to run. This is true when you execute a built-in command or run a program that you have installed. To figure out where to look for such programs, the shell searches through the directories specified by the ``PATH`` variable, which in my case is .. code-block:: none $ printenv PATH PATH=/Users/motamed/mpich/bin:/usr/sbin:/sbin:/usr/local/bin:/sw/bin:/usr/bin So if you install some software in a new location, say ``HOME/newbin``, you may want to append ``PATH`` by using the ``export`` command .. code-block:: none export PATH=$PATH:$HOME/newbin The search paths are separated by a colon. If you want the added directory to be searched first you can prepend the ``PATH`` instead .. code-block:: none export PATH=$HOME/newbin:$PATH The .bashrc or .profile files +++++++++++++++++++++++++++++++ Whenever you open a new terminal the file ``$HOME/.bashrc`` or ``$HOME/.profile`` (depending on your OS) is executed as a bash script. In this file you can put anything that you want to be executed on startup. For example, if you want to do some permanent changes (like exporting a new environment variable, such as ``HPSC`` above) or to always append the ``PATH`` variable (like we did above), you can add them as new lines to your ``.bashrc`` or ``.profile`` file. You can also premanenetly redefine the biult-in commands. For example, you can redefine ``ls`` to color the output by adding the line ``alias ls='ls -G'`` to your ``.bashrc`` or ``.profile`` file. which +++++++++++++++++++++++++++++++ The ``which`` command displays the full path to the program that is being executed: .. code-block:: none $ which python /opt/local/bin/python $ which gfortran /usr/local/bin/gfortran top +++++++++++++++++++++++++++++++ The ``top`` command displays the programs that are currently running on the system with general information about memory and processor usage. It is particularly useful to check for other users running large programs before running one yourself on a shared computer. You can also use it to monitor your own programs. To exit ``top`` simply press ``q``. If would like to kill a process, find the PID (process ID) number from ``top``, and then use the ``kill`` command in terminal window by typing ``$ kill PID_number``. Logging in to a remote computer +++++++++++++++++++++++++++++++ **ssh:** To login to a remote computer you can use the `secure shell`__, for example to login to ``linux.unm.edu`` I would type: .. code-block:: none ssh -X motamed@linux.unm.edu Here the ``-X`` starts the X-server so you can open windows locally on your machine. The result of the above command is the following message (you will need to type your UNM password, the same as the one you use to login to `my.unm`): .. code-block:: none ======================================================================== = WARNING NOTICE TO USERS = = Authorized uses only. = = All activity may be monitored and reported. = ======================================================================== motamed@linux.unm.edu's password: Last login: Thu Aug 13 15:42:19 2020 from MY.COM.PU.TER --------------------------------- linux.unm.edu or ftp.unm.edu is Red Hat Enterprise 7 --------------------------------- See available software and learn about this environment at: http://LinuxShellServices.unm.edu [motamed@comet ~]$ I am then ready to type commands after the prompt ``$`` on the remote computer. To exit the remote computer, simply type ``exit``. **scp:** The ``scp`` command is used to transfer files between two computers. For example, to copy a file named `filename1` in my local directory to a directory `dir_name` on the remote computer, I would type: .. code-block:: none $ scp filename1 motamed@linux.unm.edu:dir_name And to copy a remote file named `filename2` to my local machine, I would type: .. code-block:: none $ scp motamed@linux.unm.edu:dir_name/dir_name . where ``.`` in the end of the above command line means "`this directory`". I could instead give the path to a different local directory. ------------------- A good resource for reading more about shells and Unix/Linux is `software-carpentry`__. For a list of Unix commands with details see `here`__. __ http://en.wikipedia.org/wiki/Bash_(Unix_shell) __ https://en.wikipedia.org/wiki/C_shell __ https://en.wikipedia.org/wiki/File_system_permissions __ http://en.wikipedia.org/wiki/Secure_Shell __ https://software-carpentry.org/lessons/ __ https://en.wikipedia.org/wiki/List_of_Unix_commands