Lesson 2 – Linux Fundamentals

I can’t emphasize enough how important it is to feel comfortable in Linux/Unix if you’re going to get serious about penetration testing.  My biggest complaint while I was teaching is that there was no Linux pre-req for my course.  I did my best to spin up the newbies or refresh those who hadn’t touched Linux in a while. By the end of the course, most of my students were compotent Linux command line users. I even got a student to change her opinion from “Linux sucks!” to “Linux is okay.” 😄

• Ken Thompson (left) and Dennis Ritchie (right) were computer scientists work at AT&T Bell Labs in the 1960’s and 1970’s.  They were originally working on a joint project with MIT, GE, and others called “Multics”.  Bell Labs ended up dropping out of the project, but Ken and Dennis took what they learned and used it to develop a hobby operating system (OS), that they jokingly called “UNIX.”
• Ken’s motivation was to write an OS so that a computer game he was working on, called Space Travel, would work better on the unused PDP-7 computer they had in the lab.
• More details on Unix history can be found here, in an excerpt from the book The Art of Unix Programming: http://www.catb.org/esr/writings/taoup/html/ch02s01.html

• Ken Thompson (seated) and Dennis Ritchie, hacking away on a DEC PDP-11.  Despite being the size of a set of bookcases, the PDP-11 series were considered “minicomputers” back then…because they were smaller than the IBM mainframes that took up entire rooms.
• UNIX’s big claim to fame would be that large portions of it were eventally rewritten in the high-level C programming language (which was developed by Dennis Ritchie and others at Bell Labs), instead of the PDP-7 or PDP-11’s processor-specific assembly language.  This allowed UNIX to be easily rewritten for other platforms and contributed to its (and the C language’s) spread.
• Also, at the time, AT&T still had a monopoly on telephone service and was barred from entering the computer market by the US Department of Justice.  As part of a settlement with the US DoJ, they had to freely and cheaply license any computer software they developed at Bell Labs.
• Being cheaply distributed, in a high-level language, and with the source code included, made UNIX extremely popular in college computer science departments.
• One of the earliest customers was the University of California, Berkley.  In 1977, they released the first Berkley Software Distribution (BSD), a collection of customizations and new software for UNIX.  Eventually, the BSD project would rewrite the entire UNIX kernel and become its own operating system.  The little devil cartoon is Beastie, the BSD mascot.

• The 1980’s, Richard M. Stallman (pictured) quit his job as an MIT researcher to found the GNU Project.  The idea was to make a totally free, open-source operating system that anyone could use and contribute to, free of any corporate ownership.
• The project succeeded in developing tons of very useful tools, that many people would use with their existing UNIX operating system, such as the Bourner Again (bash) shell, the GNU Compiler Collection (gcc) and the GNU Debugger (gdb).
• However, they never managed to develop a fully-functional OS kernel to replace the proprietary AT&T-developed one. There are people still working on it and it’s called GNU Hurd. You can even download a prototype and try it out, but it’s been stuck in development hell for decades.  Even if it is ever finished into a stable product, it would still just be a curiosity.  Someone else already solved the kernel problem…

• In 1984, AT&T agreed to break up the old telephone monopoly in exchange for being allowed to finally sell computers.  It thought it could turn the popularity of UNIX into a cash cow…but failed miserably in the end.  As several other companies (such as HP, IBM, and Sun) had already developed commercial UNIX distributions, AT&T kicked off what is called the “UNIX Wars,” trying to retake the market from all these competitors with their new Unix System V (as in Roman numeral for “5”).  All the ended up happening was that the market became a mess of look-alike, but incompatible, products trying to push different, competing standards.
• In 1991, Finnish computer science student first shared his custom-built OS kernel on Usenet.  This would finally solve the problem of the missing GNU kernel and create a free and usable operating system that anyone could download, install, and even customize.  The other big innovation was that Linux was designed to run on IBM PC-compatible computers and could do the sort of advanced tasks that would normally require a very expensive UNIX workstation.

• Portable: can be easily rewritten for different processors and platforms, such as Intel x86, ARM, PowerPC, SPARC, etc.
• Multi-tasking: it can run several different programs or tasks at the same time
• Multi-user in a time-sharing configuration: it can support multiple users at the same time, who all share the system’s resources (CPU, RAM, disk space, etc.)

• Really, everything is represented by a file.  Including devices like the hard drive and CD-ROM, processes in memory, and even TCP/UDP ports
• Linux cares whether the letters in file names and commands are upper-case or lower-case

• The big surviving commercial Unix distributions you’ll see in corporate environments are IBM AIX, Oracle Solaris (developed by Sun Microsystems before Oracle bought them), and Hewlett-Packard HP-UX.  Many of the older proprietary Unices have been replaced by free and open-source lookalikes like Linux and FreeBSD.
• SCO had two Unices: OpenServer (which, believe it or not, started life as Xenix, Microsoft‘s own Unix distro) and UnixWare, which is bought from Novell.  SCO itself is dead and these two distros are now sold by a company called Xinuos
• The weird little icon in the bottom left is the logo of Silcon Graphics Inc. (SGI) IRIX, a Unix distro used on SGI Iris workstations and popular in the early 3D graphics world.  It’s been dead for some years now.
• Tru64 was the last Unix distribution produced at DEC, for the Alpha platform.  Some Alphas still get legacy support from HP (which ended up owning all of DEC’s intellectual property).  Otherwise, Tru64 is dead.

• The different BSD operating systems have always been free and open source.  Actually, if BSD hadn’t been mired in stupid lawsuits from AT&T in the late 80’s and 90’s, we’d probably be using it exclusively and Linux would’ve never happened.
• The three big BSD distributions are FreeBSD, NetBSD, and OpenBSD.  FreeBSD and NetBSD were some of the first projects to spin off the original UC Berkley work.  OpenBSD is itself a fork of NetBSD, with a strong focus on making the OS as secure as possible.  There are numerous other forks off of FreeBSD, including Dragonfly BSD, GhostBSD, PC-BSD (with a focus on being very user-friendly, especially to people migrating from Windows), and many others.
• Really, the biggest BSD-derived operating system is actually Mac OS X.
• When Steve Jobs was forced out of Apple in the mid-80’s, he founded a company called NeXT.  NeXT produced high-end workstations with an easy-to-use graphical interface, running an OS derived from BSD Unix called NeXTSTEP.  Later, Apple would purchase NeXT, turn NeXTSTEP into Mac OS X, and Jobs would make his triumphant return to the CEO chair of Apple.

• From the beginning, people were free to take the Linux kernel, the GNU tools, and any other software they wanted a throw together a “distribution.”  There was never really any “official” Linux distribution, just whatever the community put together.
• One of the big families is that of Red Hat, one of the biggest commercial Linux vendors.  Their flagship product is Red Hat Enterprise Linux (RHEL).  Fedora is their user desktop-oriented release that is maintained by a community of Red Hat employers and volunteer developers.  CentOS is a clone of RHEL for people who want the advantages of a server Linux OS but don’t want to purchase a Red Hat support contract.

• Another big and important family of distributions is the Debian family.
• Members of the family include Ubuntu and Linux Mint, which both aim to be user-friendly and easy for Windows users to switch to.
• Kali Linux is also derived from Debian.  Its predecessor, BackTrack Linux, was a fork of Ubuntu.
• Knoppix isn’t as popular as it used to be, but was a pioneer in the concept of the “LiveCD” and “LiveUSB”: an operating system that could run from a CD-ROM or USB stick without having to install it onto a hard drive.
• Tails (The Amnesic Incognito Live System) is a privacy and security-focused distro, popularly used by NSA leaker Edward Snowden, that forces all traffic to go over the Tor anonymizer network.

• Some other popylar Linux distros include:
• SuSE Linux, another commercial Linux distribution that’s sold by Novell.
• Arch and Gentoo are famous for their customizability and orientation towards power users
• Puppy Linux is geared towards older or resource-constrained system
• Slackware is one of the oldest Linux distributions
• Don’t forget that Google Android itself is a derivative of Linux, though heavily modified.

• http://www.ee.surrey.ac.uk/Teaching/Unix/
• Sorry, you’re on your own here. You can easily download the ISO file for Linux Mint or another Linux distro and try installing it in VMware or VirtualBox.  It’s not that hard, I promise! 😉

• Tab completion – if you type part of a command or file name and hit the “Tab” key on your keyboard, Unix/Linux will try to guess what you mean.  If there’s more than one possibilities, hit “Tab” twice
• Arrow keys – clicking the up or down arrow keys on your keyboard will move through the history of commands you’ve entered.  Also, some programs (such as vim) may use the H, J, K, L keys instead of the arrow keys.  This is a throwback to older keyboards from the 70’s.
• Quotes and escape characters – if you have to enter a file name that has spaces, asterisks, or other special characters (that Linux would normally use when interpretting commands), you can either wrap the name in single-quotes, double-quotes, or use the backslash ( \ ) to “escape” those characters and tell Linux to ignore them.  Examples:   ls -al “Hello World *.txt”; ls -al ‘Hello World *.txt’; ls -al Hello\ World\ \*.txt
• Scripting – just like batch or Powershell scripting in Windows, Linux shells have robust scripting languages that act like mini-programming languages.  These files might have a .sh file extension and, if you open them in a text editor, they will start with a hash symbol, an exclamation point, and the filepath of the shell they use, such as #!/bin/bash or #!/bin/sh
• Package managers (apt, yum, pacman, ports, etc.) – package managers are built into almost every Linux distro and allow you to easy pull down new software from the internet and install it.  Debian uses the apt package management system and .deb installer files.
• passwd – this is the command to change your or (if you’re an admin) another user’s password
• su and sudo – su stands for “switch user” and sudo for “switch user and do this”; these utilities allow you to login as or perform an action as another user, usually the root user. Most Linux systems, for accountability and security, don’t let admins login as root, so they use a combination or su and sudo instead to perform root-level actions, like installing new software, changing system files, creating/deleting users, etc.
• ln – this is the command to make a link, like making a shortcut in Windows
• nano, vi/vim, and emacs – these are some of the most common text editors you’ll see on Linux/Unix systems.  Nano is the easiest to use, vi/vim are de facto standards and you’ll find them on any Unix system, and emacs is a very programmer-centric text editor
• touch – this command will update the time stamp on a file or directory and is also a quick way to make a new blank file
• top – this is a command-line system monitor program, much like Task Manager in Windows, where you can see and sort running processes by RAM usage, CPU utilization, PID, and other values
• exit – this is a way of exitting or logging out of your session, you can use it to kill an ssh session or if you logged in as root using “su”, it would end root’s session and drop you back to your normal user session

• https://www.kali.org

• http://www.openvim.com
• http://www.vim.org

Lesson 1 – Introduction

• “Responsible disclosure” means that, if you find an organization’s server is misconfigured, or their software has a bug that can be exploited, etc., you tell the company about it first and give them time to remediate before you disclose anything publicly
• May also hear it called “coordinated disclosure”
• “Full disclosure” is telling anyone and everyone about a vulnerability as early as possible, not necessary giving the target org any lead time
• Many security professionals report vulnerabilities and give the company a deadline to fix it before the details will be published to a mailing list, blog, etc.

• Vulnerability – a weakness in the system design, implementation, software or code, or the lack of a mechanism
• Threat – any agent, condition, or circumstance that could potentially cause harm, loss, damage, or compromise to an IT or data asset
• Exploit – a piece of software, tool, or technique that takes advantage of a vulnerability; could lead to access, privilege escalation, denial of service, etc.
• Exposure – when a vulnerability is reachable by a threat, who could then exploit it

• Phone phreaks, much like computer hackers, were part curious explorer and part petty criminal.  They were extremely interested in how the Bell telephone system worked and poked and prodded their way through…usually with the goal of successfully committing “toll fraud” or getting free long distance calls.  Before the final divestiture of the Bell System in 1984, long distance was extremely expensive.  For example, a Bell commercial from 1970 touts the new minimum rate of 70 cents for a three minute call, which adjusted for inflation would be $4.27 today.  And that was the (cheaper) weekend rate.  Phone phreaks figured out ways to game the phone system to give themselves free long distance calls and later free features like conference calling and call waiting.  As the telephone system has become harder to cheat, and phone calls get cheaper and cheaper, phone phreaking has almost disappeared.
• Worth a read is the brief “Conscience of a Hacker” (AKA the Hacker Manifesto) by the Mentor: http://phrack.org/issues/7/3.html

• I could go back even further to Samuel Morse and the invention of the electric telegraph in 1837, but the telephone was a much more democratized invention.  Telegraphs were in special offices, controlled by specialist operators; telephones were in your house and you could use them whenever you wanted.  You could draw a parallel between mainframes and home PCs, maybe.
• ARPANET was named for DARPA, the Defense Advanced Research Projects Agency, which funds new inventions for the US military.  Kind of like Q Branch from the James Bond novels and movies 😉
• The telephone network laid the global infrastructure that would allow ARPANet (and other research networks) to involve into the Internet and interconnect computer systems across the planet.
• ARPANET is also important for setting TCP/IP as the standard protocol stack for the future Internet.

• John Draper discovered that a toy whistle (that came as the free prize in a box of Cap’n Crunch cereal) produced the right 2600 hertz tone needed to open new trunk lines.  He used it as inspiration to build a tone-generating device for making free phone calls.  Steve Jobs and Steve Wozniak famously made and sold blue boxes before going on to found Apple.  Before blue boxes, some people had actually figured out how to whistle the right tones to fool the phone system.
• BBS’s were like early forums, where members could post messages, send and receive email, share files, play games, etc.  This was before the Internet was available in people’s homes.  To reach a BBS, you had to dial its telephone number via your computer’s modem.  If the computer hosting the BBS only had one phone line connected to it, only one person could dial in and use it at a time.  If you dialed up a BBS in another city, you’d be paying long-distance rates by the minute to use it.  Much like today, BBS’s could also be havens for illegal file sharing, cracking copy-protected software, distributing porn, and other shady activities.
• This actually created a natural progression from BBS user to phreaker to hacker.  Take for instance a kid who gets a modem and discovers a few BBS’s in his local area.  These lead him to the numbers of other BBS’s in other cities.  He calls them up, makes friends, plays cool games, shares files, and has a good time…until his parents’ phone bill comes!  After his parents get done grounding him, he probably sulks on the local boards, complaining about how he misses the new friends he made on those other boards and how stupid long-distance phone rates are.  A phreaker on the BBS tell him how to build a bluebox, or shares some stolen phone card #’s with him, so he can get back on those far-away BBS’s without any pesky long distance charges.  Our kid has just transitioned into a phone phreaker.  The next step is learning about you can sneak into corporate computers and look at all the cool stuff they have…maybe break into a game company’s computer and get a free copy of the hottest DOS game…or break into your school’s computer and change your grades!  Our teenager has finally made the leap to hacker.
• Usenet is like a hybrid between email and internet forums, but very decentralized and spread out over multiple servers.  Like BBS’s, it was a popular way for computer hobbyists, hackers, and others to connect…but since Usenet was mostly run on big iron at research labs, universities, and corporations, it’s user base wasn’t as big.  Crusty old Usenet denizens used to hate the month of September, when they’d have to deal with all the n00b university students who just got their computer accounts and started stumbling into Usenet like idiots.  Once Usenet became available to home internet users in the 90’s, the crusty old Usenetters referred to it as “eternal September” because of the constant stream of newbies.  Usenet is still around, via Google and Yahoo Groups.  Paid usenet accounts are also popular for getting access to private newsgroups where you can pirate software, movies, TV shows, etc.  Probably won’t be long before the authorities start cracking down on it, like Napster and torrents before it.

• CNN recently did a video short on the 414 Gang: http://www.cnn.com/videos/tech/2015/03/10/digital-shorts-original-teenage-hackers-orig.cnn

• US-CERT: US Computer Emergency Response Team
• Some consider the 80’s the golden age of hacking. Many of the first hackers got their start in this era, whether through BBS’s, online services like CompuServe, or their university’s Unix mainframe and connection to the ARPANet.
• A great book on this era, which I highly recommend, is Bruce Sterling’s The Hacker Crackdown, which you can buy on Amazon or read it for free online: http://www.gutenberg.org/ebooks/101

• EFF: Electronic Frontier Foundation
• ISP: Internet Service Provider
• DEC: Digital Equipment Corporation, makers of the influential PDP-11 and VAX computer lines that Unix grew up on.  Sadly no more, was bought out by Compaq in the late 1990’s, which was in turn bought by HP in 2002.

• Mitnick’s escapes are well-documented.  “Takedown,” by John Markoff and Tsutomu Shimomura, was written by some of the people who caught him the second time.  Mitnick has written several of his own books about hacking, including “Ghost in the Wires,” his account of his years on the run from the law.
• DDoS: Distributed Denial-of-Service
• Mitnick spent part of his imprisonment in solitary confinement after some of his underground enemies convinced authorities that he could start a nuclear war just by whistling into a pay phone.
• Mitnick has written several books on his experiences, such as The Art of Intrusion and Ghost in the Wires

• The biggest breach of 2013 was definitely the NSA 😉

• Script kiddie (or skiddie, for short) is a derogatory term for a cybercriminal who doesn’t know what the f**k he’s doing.  The stereotype is a minor who only knows how to download and clicky-clicky on attack tools he finds on hackforums.net.  In reality, the majority of “cybercriminals” fit in this category.  Most are just petty criminals in Russia or other non-extradition countries who run exploit kits they bought on the darknet and steal bank account credentials and credit card numbers.
• APT, or advanced persistent threat, is code-word for nation-sponsored hackers (Chinese PLA; Russian FSB, MVD, or GRU; American NSA, CYBERCOM; British GCHQ).  Lots of money, lots of talent, and lots of time on their hands to hack who and what they want.
• The whole “cracker” deal has to do with white hats who feel that the term “hacker” has been misappropriated by the media and taken on a purely negative connotation.  They feel that “hacker” should still be used in the old sense, as in a skilled programmer or a technical genius, not a fat Russian cybercriminal who’s stealing credit card numbers by phishing your grandma on Facebook.
• It’s mostly a losing battle.

• Hacktivists: a sub-class of gray hat; they may violate the law by breaking into a company without permission, but they were motivated to do it to uncover evidence that a company has been bribing politicians or dumping toxic waste in rivers, for example.
• “Suicide hacker” is a stupid term you might see on the test.  Basically a hacker that doesn’t care if they get caught or if they hurt people.  I’ve never heard it used outside of EC-Council material.

• Money is an obvious motive (CC #’s, identity theft material like SSNs or other personal info, bank credentials, etc.)
• Spies and cyberwarfare operatives are especially attracted to stealing sensitive info, like classified data, trade secrets, government communications, etc.
• Some hackers are motivated by ideology, like Wikileaks contributors, some Anonymous or patriotic hackers like th3j3st3r
• Some hackers pop boxes for street cred and respect within the underground
• And some hackers are just a**holes who want to troll, damage sites, or commit the digital equivalent of vandalism, like many other Anons or Lizard Squad

• CFAA is the grand-daddy of hacking laws, has been expanded several times since first passing in 1986
• ECPA expanded the laws that protect phone conversations to also include electronic communications, such as email, HTTP, etc.
• CALEA expanded wiretapping laws to cover the Internet
• DMCA imposes penalties for cracking DRM (digital rights management) or making and distributing tools for that purpose
• CSEA was an amendment to the USA PATRIOT Act to make it easier for government to tap Internet lines and request information from ISPs in the name of homeland security

• I always be hackin’ in a tie and ski mask!
• In fact, if you want to find out who’s attacking your company, just hack back the IP that was attacking you and look at them through the web cam.  If they’re wearing a dark hoodie, balaclava, or burglar mask, then you know you have the right guy!  #protip

• Shrink-Wrap Code: code that gets reused across lots of different software.  Prime examples would be the OpenSSL code libraries or the Cocoa Touch framework that all iOS apps build upon.  A weakness in a common library or method like that expands the attack surface considerably.

• These are the phases an attack by cybercriminal, APT, hacktivist, or other malicious actor

• Audits are overwhelmingly concerned with making sure the right policies and procedures are in place and that they are being followed
• Vulnerability assessments go as far as finding the weaknesses, but not actively attempting to exploit them
• A penetration test goes all the way, finds the vulnerabilities AND exploits them to demonstrate what an attacker could do

• White box: Full Knowledge.  You have complete information on the network, ranges, systems, infrastructure, etc.  Usually used to simulate an insider attack.
• Black box: No Knowledge.  You might not know anything except the name of the company and have to discover/reconnoiter everything yourself.  Usually used to simulate an outside attack.
• Gray box: Partial Knowledge.  Somewhere in the middle; client gives you some information, but not everything.  This is the most common sort of pentest.

• E&O = Errors & Omissions

• This will all be discussed in more depth in future lessons.

• Threat modeling is a little outside the scope of this class.  The overall point is to focus your attacks or defense based on what sort of people and/or organizations threaten you.  Obviously, if you’re trying to defend against APT, your threat model is going to be extremely different from someone whose only threat is script kiddies defacing their website.

• The best way to experiment with Kali Linux is to download the official VM or the ISO file and run it inside virtualization software.  If you’re on Windows or Linux, VMware or Virtualbox are both free and work great.  If you’re on Mac, Virtualbox is the only free option available to you.  There are numerous tutorials and instructions on how to install it on the Kali Linux website.