I can’t emphasize enough how important it is to feel comfortable in Linux/Unix if you’re going to get serious about penetration testing. My biggest complaint while I was teaching is that there was no Linux pre-req for my course. I did my best to spin up the newbies or refresh those who hadn’t touched Linux in a while. By the end of the course, most of my students were compotent Linux command line users. I even got a student to change her opinion from “Linux sucks!” to “Linux is okay.” 😄
- Ken Thompson (left) and Dennis Ritchie (right) were computer scientists work at AT&T Bell Labs in the 1960’s and 1970’s. They were originally working on a joint project with MIT, GE, and others called “Multics”. Bell Labs ended up dropping out of the project, but Ken and Dennis took what they learned and used it to develop a hobby operating system (OS), that they jokingly called “UNIX.”
- Ken’s motivation was to write an OS so that a computer game he was working on, called Space Travel, would work better on the unused PDP-7 computer they had in the lab.
- More details on Unix history can be found here, in an excerpt from the book The Art of Unix Programming: http://www.catb.org/esr/writings/taoup/html/ch02s01.html
- Ken Thompson (seated) and Dennis Ritchie, hacking away on a DEC PDP-11. Despite being the size of a set of bookcases, the PDP-11 series were considered “minicomputers” back then…because they were smaller than the IBM mainframes that took up entire rooms.
- UNIX’s big claim to fame would be that large portions of it were eventally rewritten in the high-level C programming language (which was developed by Dennis Ritchie and others at Bell Labs), instead of the PDP-7 or PDP-11’s processor-specific assembly language. This allowed UNIX to be easily rewritten for other platforms and contributed to its (and the C language’s) spread.
- Also, at the time, AT&T still had a monopoly on telephone service and was barred from entering the computer market by the US Department of Justice. As part of a settlement with the US DoJ, they had to freely and cheaply license any computer software they developed at Bell Labs.
- Being cheaply distributed, in a high-level language, and with the source code included, made UNIX extremely popular in college computer science departments.
- One of the earliest customers was the University of California, Berkley. In 1977, they released the first Berkley Software Distribution (BSD), a collection of customizations and new software for UNIX. Eventually, the BSD project would rewrite the entire UNIX kernel and become its own operating system. The little devil cartoon is Beastie, the BSD mascot.
- The 1980’s, Richard M. Stallman (pictured) quit his job as an MIT researcher to found the GNU Project. The idea was to make a totally free, open-source operating system that anyone could use and contribute to, free of any corporate ownership.
- The project succeeded in developing tons of very useful tools, that many people would use with their existing UNIX operating system, such as the Bourner Again (bash) shell, the GNU Compiler Collection (gcc) and the GNU Debugger (gdb).
- However, they never managed to develop a fully-functional OS kernel to replace the proprietary AT&T-developed one. There are people still working on it and it’s called GNU Hurd. You can even download a prototype and try it out, but it’s been stuck in development hell for decades. Even if it is ever finished into a stable product, it would still just be a curiosity. Someone else already solved the kernel problem…
- In 1984, AT&T agreed to break up the old telephone monopoly in exchange for being allowed to finally sell computers. It thought it could turn the popularity of UNIX into a cash cow…but failed miserably in the end. As several other companies (such as HP, IBM, and Sun) had already developed commercial UNIX distributions, AT&T kicked off what is called the “UNIX Wars,” trying to retake the market from all these competitors with their new Unix System V (as in Roman numeral for “5”). All the ended up happening was that the market became a mess of look-alike, but incompatible, products trying to push different, competing standards.
- In 1991, Finnish computer science student first shared his custom-built OS kernel on Usenet. This would finally solve the problem of the missing GNU kernel and create a free and usable operating system that anyone could download, install, and even customize. The other big innovation was that Linux was designed to run on IBM PC-compatible computers and could do the sort of advanced tasks that would normally require a very expensive UNIX workstation.
- Portable: can be easily rewritten for different processors and platforms, such as Intel x86, ARM, PowerPC, SPARC, etc.
- Multi-tasking: it can run several different programs or tasks at the same time
- Multi-user in a time-sharing configuration: it can support multiple users at the same time, who all share the system’s resources (CPU, RAM, disk space, etc.)
- Really, everything is represented by a file. Including devices like the hard drive and CD-ROM, processes in memory, and even TCP/UDP ports
- Linux cares whether the letters in file names and commands are upper-case or lower-case
- The big surviving commercial Unix distributions you’ll see in corporate environments are IBM AIX, Oracle Solaris (developed by Sun Microsystems before Oracle bought them), and Hewlett-Packard HP-UX. Many of the older proprietary Unices have been replaced by free and open-source lookalikes like Linux and FreeBSD.
- SCO had two Unices: OpenServer (which, believe it or not, started life as Xenix, Microsoft‘s own Unix distro) and UnixWare, which is bought from Novell. SCO itself is dead and these two distros are now sold by a company called Xinuos
- The weird little icon in the bottom left is the logo of Silcon Graphics Inc. (SGI) IRIX, a Unix distro used on SGI Iris workstations and popular in the early 3D graphics world. It’s been dead for some years now.
- Tru64 was the last Unix distribution produced at DEC, for the Alpha platform. Some Alphas still get legacy support from HP (which ended up owning all of DEC’s intellectual property). Otherwise, Tru64 is dead.
- The different BSD operating systems have always been free and open source. Actually, if BSD hadn’t been mired in stupid lawsuits from AT&T in the late 80’s and 90’s, we’d probably be using it exclusively and Linux would’ve never happened.
- The three big BSD distributions are FreeBSD, NetBSD, and OpenBSD. FreeBSD and NetBSD were some of the first projects to spin off the original UC Berkley work. OpenBSD is itself a fork of NetBSD, with a strong focus on making the OS as secure as possible. There are numerous other forks off of FreeBSD, including Dragonfly BSD, GhostBSD, PC-BSD (with a focus on being very user-friendly, especially to people migrating from Windows), and many others.
- Really, the biggest BSD-derived operating system is actually Mac OS X.
- When Steve Jobs was forced out of Apple in the mid-80’s, he founded a company called NeXT. NeXT produced high-end workstations with an easy-to-use graphical interface, running an OS derived from BSD Unix called NeXTSTEP. Later, Apple would purchase NeXT, turn NeXTSTEP into Mac OS X, and Jobs would make his triumphant return to the CEO chair of Apple.
- From the beginning, people were free to take the Linux kernel, the GNU tools, and any other software they wanted a throw together a “distribution.” There was never really any “official” Linux distribution, just whatever the community put together.
- One of the big families is that of Red Hat, one of the biggest commercial Linux vendors. Their flagship product is Red Hat Enterprise Linux (RHEL). Fedora is their user desktop-oriented release that is maintained by a community of Red Hat employers and volunteer developers. CentOS is a clone of RHEL for people who want the advantages of a server Linux OS but don’t want to purchase a Red Hat support contract.
- Another big and important family of distributions is the Debian family.
- Members of the family include Ubuntu and Linux Mint, which both aim to be user-friendly and easy for Windows users to switch to.
- Kali Linux is also derived from Debian. Its predecessor, BackTrack Linux, was a fork of Ubuntu.
- Knoppix isn’t as popular as it used to be, but was a pioneer in the concept of the “LiveCD” and “LiveUSB”: an operating system that could run from a CD-ROM or USB stick without having to install it onto a hard drive.
- Tails (The Amnesic Incognito Live System) is a privacy and security-focused distro, popularly used by NSA leaker Edward Snowden, that forces all traffic to go over the Tor anonymizer network.
- Some other popylar Linux distros include:
- SuSE Linux, another commercial Linux distribution that’s sold by Novell.
- Arch and Gentoo are famous for their customizability and orientation towards power users
- Puppy Linux is geared towards older or resource-constrained system
- Slackware is one of the oldest Linux distributions
- Don’t forget that Google Android itself is a derivative of Linux, though heavily modified.
- http://www.ee.surrey.ac.uk/Teaching/Unix/
- Sorry, you’re on your own here. You can easily download the ISO file for Linux Mint or another Linux distro and try installing it in VMware or VirtualBox. It’s not that hard, I promise! 😉
- Tab completion – if you type part of a command or file name and hit the “Tab” key on your keyboard, Unix/Linux will try to guess what you mean. If there’s more than one possibilities, hit “Tab” twice
- Arrow keys – clicking the up or down arrow keys on your keyboard will move through the history of commands you’ve entered. Also, some programs (such as vim) may use the H, J, K, L keys instead of the arrow keys. This is a throwback to older keyboards from the 70’s.
- Quotes and escape characters – if you have to enter a file name that has spaces, asterisks, or other special characters (that Linux would normally use when interpretting commands), you can either wrap the name in single-quotes, double-quotes, or use the backslash ( \ ) to “escape” those characters and tell Linux to ignore them. Examples: ls -al “Hello World *.txt”; ls -al ‘Hello World *.txt’; ls -al Hello\ World\ \*.txt
- Scripting – just like batch or Powershell scripting in Windows, Linux shells have robust scripting languages that act like mini-programming languages. These files might have a .sh file extension and, if you open them in a text editor, they will start with a hash symbol, an exclamation point, and the filepath of the shell they use, such as #!/bin/bash or #!/bin/sh
- Package managers (apt, yum, pacman, ports, etc.) – package managers are built into almost every Linux distro and allow you to easy pull down new software from the internet and install it. Debian uses the apt package management system and .deb installer files.
- passwd – this is the command to change your or (if you’re an admin) another user’s password
- su and sudo – su stands for “switch user” and sudo for “switch user and do this”; these utilities allow you to login as or perform an action as another user, usually the root user. Most Linux systems, for accountability and security, don’t let admins login as root, so they use a combination or su and sudo instead to perform root-level actions, like installing new software, changing system files, creating/deleting users, etc.
- ln – this is the command to make a link, like making a shortcut in Windows
- nano, vi/vim, and emacs – these are some of the most common text editors you’ll see on Linux/Unix systems. Nano is the easiest to use, vi/vim are de facto standards and you’ll find them on any Unix system, and emacs is a very programmer-centric text editor
- touch – this command will update the time stamp on a file or directory and is also a quick way to make a new blank file
- top – this is a command-line system monitor program, much like Task Manager in Windows, where you can see and sort running processes by RAM usage, CPU utilization, PID, and other values
- exit – this is a way of exitting or logging out of your session, you can use it to kill an ssh session or if you logged in as root using “su”, it would end root’s session and drop you back to your normal user session
Dan Helton's Blog 