Summary and Schedule

Command line interface (CLI) and graphic user interface (GUI) are different ways of interacting with a computer’s operating system. They have different pros and cons. Most people are familiar with the GUI as it is the default interface for most software, particularly on Windows and Mac OS. When using the GUI, you see and interact with visual representations of files, folders, applications, and most other functions of your computer. When using the CLI, you work largely with text representations of software, files, folders, input and output. The shell is a program that allows you to control your computer by typing instructions on the CLI with a keyboard.

There are several reasons to learn how to use the CLI:

• For most bioinformatics tools, there are no graphical interfaces. If you want to work in metagenomics or genomics, you’re going to need to use the CLI/ shell.
• The shell gives you power. The command line allows you to work more efficiently. Tasks that are repetitive (e.g. renaming hundreds of files) can be automated. Tasks that are tedious (e.g. testing a range of input parameters) can be simplified.
• To use remote computers or cloud computing, you often need to use the shell.

Register for this workshop and explore Unix with the Computing Basics team.

Getting Started

This lesson assumes no prior experience with the tools covered in the workshop. However, learners are expected to have some familiarity with biological concepts. Participants should bring their laptops and plan to participate actively.

For Instructors

If you are teaching this lesson in a workshop, please see the Instructor notes.

	Setup Instructions	Download files required for the lesson
Duration: 00h 00m	1. Introducing the Shell	What is a command shell and why would I use one? How can I move around on my computer? How can I see what files and directories I have? How can I specify the location of a file or directory on my computer?
Duration: 00h 20m	2. Navigating Files and Directories	How can I perform operations on files outside of my working directory? What are some navigational shortcuts I can use to make my work more efficient?
Duration: 01h 10m	3. Working with Files and Directories	How can I view and search file contents? How can I create, copy and delete files and directories? How can I control who has permission to modify a file? How can I repeat recently used commands?
Duration: 01h 55m	4. Redirection	How can I search within files? How can I combine existing commands to do new things?
Duration: 02h 40m	5. Writing Scripts and Working with Data	How can we automate a commonly used set of commands?
Duration: 03h 20m	6. Bioinformatics examples	How can we work with files specific to the biological sciences in Unix?
Duration: 03h 32m	Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.

Setup

To participate in this lesson, you will need a working Unix-like shell environment. We will be using Bash (Bourne Again Shell) which is standard on Linux and macOS. Some macOS users (Catalina or later) will have zsh (Z shell) as their default version. Even if you are a Windows user, learning Bash will open up a powerful set of tools on your personal machine, and familiarize you with the standard remote interface used on most servers and supercomputers.

For the workshop, we will use the Broad’s login servers. To access those machines, your laptop needs to be on the Broad-Internal wireless network when you are on-site OR on the Broad VPN if you have a non-Broad-issued computer or are off-site.

Prerequisite

Virtual Private Network (VPN) Access

To reach to the Broad login servers when you are off-site or using a non-Broad computer, you will connect to the Broad VPN to access internal Broad resources.

VPN Installation

• Use Broad Self Service (Mac or Windows) to install the Cisco AnyConnect Secure Mobility Client software

  OR

• Visit https://vpn.broadinstitute.org and follow the prompts.

You will also need to set up DUO two-factor authentication or Google 2FA to use the Broad VPN. Contact BITS for troubleshooting if you have a Broad-issued machine.

If you can’t access the documentation linked above or can’t connect to the Broad VPN using your Broad credentials, please let the workshop organizers know so we can assess whether you will be able to participate in the workshop.

Prerequisite

Terminal Setup

Bash is the default shell on most Linux distributions and older versions of macOS. Windows users will need to install WSL to provide a Unix-like environment.

Linux

Mac

Windows

The default shell is usually Bash, but if your machine is set up differently you can run it by opening a terminal and typing bash followed by the enter key. There is no need to install anything. Look for Terminal in your applications to start the Bash shell.

Open Terminal from /Applications/Utilities or Spotlight Search. In versions before Catalina, Bash is the default shell, so you do not need to do anything further. In Catalina and onwards, the default shell is zsh, which is similar but may behave differently from Bash in some cases. To switch to Bash, enter the command bash in your terminal window followed by the enter key.

On Windows, CMD or PowerShell are normally available as the default shell environments. These use a syntax and set of applications unique to Windows systems and are incompatible with the more widely used Unix utilities. However, a Bash shell can be installed on Windows to provide a Unix-like environment. For this lesson we suggest using Windows Subsystem for Linux (WSL). Open PowerShell or Windows Command Prompt in administrator mode by right-clicking and selecting “Run as administrator”, enter wsl –install, then restart your machine.

Alternatively, you can install Git Bash, part of the Git for Windows packagee

• Download the latest Git for Windows installer.
• Double click the .exe file to run the installer (for example, Git-2.42.0.2-64-bit.exe) using the default settings.
• Once installed, open the shell by selecting Git Bash from the start menu (in the Git folder).

You can also run Bash commands on a remote computer or server that already has a Unix Shell from your Windows machine. This can be done through a Secure Shell (SSH) client. One client available for free for Windows is PuTTY, which is also available through the Microsoft store.

If you encounter issues, the Carpentries has a Configuration Problems and Solutions wiki page that may help.

Broad recommends the SecurtCRT terminal application for Broad-issued Windows machines. Broadies can download through the Windows 10 Software Center. : 1. Click on the Windows logo. 1. Click on All Programs > Microsoft System Center > Configuration Manager > Software Center 1. Locate and select SecureCRT, then click on “Install”.

For help with installation on Broad-issued Windows machines, check the troubleshooting section or contact BITS.

Testing Broad Login Server access

If you are on-site, connect to the Broad-Internal wireless network. If you are off-site, connect to the Broad VPN.

Mac

Linux

Windows - WSL

Windows - SecureCRT

1. Open Terminal from /Applications/Utilities or Spotlight Search.
2. Type ssh <username>@login.broadinstitute.org [Example: ssh jcase@login.broadinstitute.org ]
3. Enter your Broad password when prompted.

1. Most Linux systems use Ctrl-Alt-T to start a Linux Terminal session.
2. On the command line, type ssh <username>@login.broadinstitute.org [Example: ssh jcase@login.broadinstitute.org ]
3. Enter your Broad password when prompted.

1. Open WSL from the start menu item or by typing wsl from a command prompt or PowerShell.
2. Type ssh <username>@login.broadinstitute.org [Example: ssh jcase@login.broadinstitute.org ]
3. Enter your Broad password when prompted.

1. Launch SecureCRT from the Start menu
2. Go to the File menu and select “Connect”. Select the “New Session”” button.
3. Within the New Session Wizard window, select SSH2 at the Protocol toggle and click Next.
4. Within the Hostname field, enter login.broadinstitute.org. Leave the Port and Firewall fields set to their defaults. Within the username field, enter your Broad username and click Next.
5. Leave the SecureFX protocol field as is (eg. SFTP), click Next.
6. Leave the session name field as it is, or rename it to something shorter. Click the Finish button.
7. X11 setup (for future reference, not needed for this workshop): Select the newly created connection entry and click the Properties icon button. The Session Options window will appear. Select Remote/X11 from the menu on the left, check Forward X11 packets. Click the OK button to close the window.
8. Finally, select your session and click the Connect button. Enter your password to log on.

Show your successful access

1. Type touch /broad/hptmp/computing_basics/<username>_was_here
2. Type ls /broad/hptmp/computing_basics Did you leave your mark?
3. Type exit to leave the server.

If you were unable to access the Broad login servers, please let the workshop organizers know so we can help you troubleshoot before the start of the workshop.