Category Archives: Pentesting

My AWAE / OSWE Experience

**UPDATE 7/17/2020**: OffSec has just updated the AWAE with a few more modules and lab machines, but at the same price point.

Back around Christmas, I took advantage of a rare sale to sign up for Offensive Security’s Advanced Web Attacks and Exploitation (AWAE) course, which is the training for the Offensive Security Web Expert (OSWE) certification. Just last week, I got my congratulatory email letting me know that I’d passed the exam and earn the cert!

This is a course I’ve been wanting to take for a very long time. Previously, it was only available as in-person, which usually meant you had to race everyone to sign up for it at Black Hat in Vegas or you had to pay a hefty premium to bring OffSec trainers in-house. I tried to get them in-house several years ago at a previous employer, but the price tag was too high for my management then. I do know of several lucky bastards at another consulting firm who got to take it privately in the US Virgin Islands years ago. But thankfully for the rest of us, the long-standing rumors finally proved true and OffSec decided to offer it as an online course last year.

So What Was AWAE Like?

The course is a very different experience from PWK / OSCP, I’m assuming that PWK hasn’t changed that much since the big update recently. In PWK, you’re put into a shared lab environment with many other students, given minimal guidance, and told to go nuts. It’s very much a “throw you into the deep end of the pool” approach to teaching you how to swim.

By contrast, AWAE is much more guided. You get your own lab environment via VPN, no having to share with other folks and worry about screwing up someone else’s work or having someone reset the machine you’re working on mid-exploit. Your lab consists of a small number of VM’s hosting web apps, all of which you have root credentials to right off the bat! That might strike you as strange, but the point here is to teach you how to do code-assisted web app pentesting, and the easiest way to do that is to give you an application where you can read its source code and debug it as it’s running. The VMs represented a fairly diverse range of web apps and technologies, including PHP, .NET, Java/Tomcat, MySQL, PostgreSQL, and Node, with both Windows and Linux web servers represented.

The course PDF and videos will run you through each application, showing you exactly where the flaws are in the code, why they work the way they do, how to debug them, and how to build a proof of concept script to exploit them. You’ll learn about using an editor to set breakpoints, decompiling applications to see the original source, turning on logging in the relevant SQL engine so you can see just what’s happening on the backend when you send an SQL injection payload, and more. The end goal of each module/machine is to get remote code execution (RCE) and a shell on the box. Along the way, they cover a pretty good gamut of common web vulnerabilities, including XSS/CSRF, code injection, bad crypto implementations, lots of blind SQL injection, and deserialization.

But it isn’t all easy step-by-step instruction-following. While the PDF and videos will show you exactly how to write a proof of concept (usually in Python or sometimes JavaScript) to exploit the main vulnerability being taught, every module has various “Extra Mile” challenges scattered throughout. The “Extra Mile” challenges provided some additional difficulty, and will ask you to do such things as find additional vulnerabilities in the code and exploit them, rewrite your proofs of concept to take advantage of a different technique, etc.

Because of the much more guided nature, and probably because I’ve already been doing web app pentesting for several years now, I only signed up for 30 days of lab time and I found it very doable in that time frame. You can, of course, sign up for longer lab times or extend your time at any point. But whereas I’d tell anyone doing OSCP to just go ahead and buy the full 90 days, this one can be done in 30 days if you have time every evening and weekend to devote to it.

So What Was the Exam Like?

I was done with the labs by February, but I had to schedule all the way out into May to find a block of days that I liked. The exam/lab access time is 48 hours (ok, technically 47 hours and 45 minutes, but whatever) long, plus an additional 24 hours to write and submit your report. I chose a block starting at 8 AM on a Thursday, so I could have that day and Friday to do the exam, Saturday for report writing, and Sunday to rest before having to go back to work.

Like PWK, this exam is now proctored. This means you’ll have to use a webcam so they can see you and the room you’re in the whole time (though they aren’t listening to you, so a microphone isn’t required). About fifteen minutes before your exam starts, you should login to the proctoring website with the credentials OffSec will provide you, install a browser extension, and share your screen with them the whole time. As you’re probably using a Windows PC or Mac and just running Kali in a VM, you’ll need to do this from the host OS. You’ll share all your screens and they’ll be watching to make sure you aren’t cheating or violating the exam rules. Whenever you need to take a break, you just have to send them a message in the chat of the proctoring website when you leave and when you return. It’s a little weird knowing you’re being watched the whole time, but it didn’t really bother me that much or impact how I would’ve done the exam otherwise.

Before you start, you’ll also have to show them a government-issued photo ID via your webcam and they’ll probably ask you to slowly pan the room so they can see and make sure no one else is in there with you helping you cheat. I hope you buy a better webcam than I did; the cheap Logitech one I got on Amazon to sit over my external monitor couldn’t focus on my ID, so I had to open the lid on my docked Macbook Pro and use it’s webcam in order to prove my identity and get started.

Once you start the exam and connect via the VPN, you’ll be in a private lab network with your challenges, much like in the OSCP exam. The challenge VMs are there for you you to break into and find flags to prove you breached them, also like the OSCP exam. But for each challenge VM, there will also be a debug VM with the same software that you’ll be given SSH/RDP credentials for; these are for you to review the code on, debug, develop your POCs, and eventually use against the challenge VMs to hack them for real. You’ll also be given access to a spare Kali VM inside the exam network. If your VPN connection is really slow or you run into any latency issues with your exploits, you can run them from this spare VM instead. One important caveat: the spare Kali VM in the lab has the default collection of Kali tools, but it doesn’t have an internet connect. So if your POC is using any exotic libraries or packages, you’ll have to SFTP them over and install them yourself.

Each challenge has a means of bypassing authentication/authorization to allow you to access it as a privileged web app user. Doing that will reveal a local.txt hash value that you can submit for points. After that, there is an RCE vulnerability somewhere in the app that will allow you to get a shell on the box, at which point you’ll be able to dump the contents of a proof.txt file for more points. There’s a total of 100 points available in the exam lab, with 85 points considered passing. In addition to submitting the hash values, you have to write a detailed report showing your work and explaining how you found and exploited the vulnerabilities, the source code for your exploit proof of concept, and screenshots showing the proof hashes. Your exploit code can be in any language you like, but realistically most people are going to be using Python, as it’s the script most commonly used in the course material itself. I did all of my exploit code in Python for the course and exam.

This exam felt very different than the OSCP exam. For one thing, the 48-hour time limit felt a little less imposing than OSCP’s 24 hours. At least for me, I didn’t feel as bad about taking breaks to rest, eat, sleep, de-stress, etc. Another thing that felt different is just the nature of code-assisted pentesting. When you’re doing an OSCP-style network pentest, you’re basically feeling around blindly in the dark, looking for a secret passage to treasure. You’re scanning and enumerating everything, trying to find what kind of software is there, what the version is, if that software/version has a known exploit against it, if it has default credentials enabled on it, etc. A lot of “black box” testing ends up being throwing shit against the wall and seeing whether it sticks or not. In a code-assisted test, all the knowledge is right there in front of you, nothing is hidden…it’s up to you to put the pieces together, find where the flaw is, and exploit it. In that respect, it feels much more like it’s your own fault if you don’t find the holes in the code.

My first day of exam-taking was stressful. I’d already told myself I’d switch to other challenges after a few hours of looking at one and getting nowhere, and that I would take regular breaks and not burn myself out. I followed my own advice, but by the evening I had no points and no clue how to progress. I’d actually resigned myself that I was probably going to fail. But I wasn’t just going to give up and quit right there, I decided I would still try to learn something from it, even if I failed. Strangely, this embrace of the possibility of failure cleared my head and lifted some of the stress of me. I approached the challenges more clearly and calmly, and by about 1 AM, I’d made some serious progress!

At that point, I crawled into bed and set an alarm for the morning. The next day, I kept grinding at, still feeling certain I wouldn’t get enough points to pass. To her credit, my wife always believed me and kept reassuring me, “You’re going to ace it!” By the late evening, I’d actually solved enough to get a passing score. I was still a little nervous about just getting the bare minimum, so I decided to look at it a little longer. To my delight, the RCE route for the final challenge jumped out at me immediately and my first attempt at a payload worked! I spent the next few hours cleaning up my exploit code, running it several times against the freshly-reverted challenge VMs just to make sure it always worked, taking all the necessary screenshots needed for the report, double-checking the proof hashes and that I’d submitted them on the control panel page correctly. Around midnight, certain I’d done everything I needed to while I still had VPN access to the exam lab, I packed it in and went to bed.

I spent the next day writing the report. I feel like this report took longer to write than the OSCP one, despite having fewer overall challenges, as I went into a lot more detail on them, showing how I deciphered the code and traced it to find the various vulnerabilities and exploit them. That evening, I zipped it up and submitted it, as per the exam instructions.

Several days of nervous email-checking later, I finally got the congratulations email!

Was AWAE / OSWE Worth It?

Unequivocally, yes! Even for someone who’s been doing web app pentesting for years, I got a tremendous amount of value out of it and it’s really inspired me to approach my own code-assisted pentests differently.

At my current job, it’s maybe 50/50 whether a client will give us source code access or not on an engagement. When we do get source, I don’t think I’ve ever seen anyone ask for actual SSH or RDP access to the web server that the test instance is running on. A lot of the times, our source access is via being added to a client’s Github repos. I have noticed there that many clients’ repos have Dockerfiles, docker-compose.yml files, and instructions for spinning up containers to run the applications.

This course has inspired me, going forward, to start asking clients for either access to the actual servers or help with building a container of the application, so I can do the sort of hands-on debugging and testing I was able to do in the AWAE course. It really makes a difference when you’re trying to build exploits for frustrating stuff like blind SQL injection.

For someone who isn’t as experienced in web app pentesting as me, this is a great introduction to doing code-assisted tests. However, one thing it doesn’t teach you is a methodology for searching/fuzzing for vulnerabilities when you don’t have source code access. So if you’re interested in doing closed source bug bounties, this course might not help you out that much.

AWAE/OSWE also covers such different ground than PWK/OSCP that having that certificate isn’t really a prerequisite for taking it. There was one part of my exam where I felt like my previous OSCP and network pentesting knowledge helped me out, but otherwise the knowledge needed for the exam is all in the domain of what you’re taught in the course.

Tips for AWAE / OSWE Success

To be honest, I didn’t really do a lot of extracurricular reading. I read the course PDF, watched all the videos, and did all the “Extra Mile” challenges, and that proved to be enough for me. But I might be a different case because I do have a few years of web app pentesting under my belt already. If you’re coming into this extremely new, I’m not sure what to recommend. Wetw0rk’s GitHub repo and Z-r0crypt’s blog are the only two outside resources that I saw recommended and I briefly looked at, so you might want to check those out too. Other than that, here is some advice I’d offer to anyone wanting to take the AWAE course and OSWE exam…

Do all the “Extra Mile” challenges: this is probably the best way to push yourself further and get ready for the sorts of challenges that the OSWE exam is going to throw your way. The AWAE forums are really helpful here if you get stuck, because the “no spoilers” policy means people won’t give away the answer, but reading posts usually gives you just enough clues to figure out whether you’re heading in the right direction, if you need to read a little more on specific subject to finally grasp the answer, or if you’re likely off on a wild goose chase and need to change your approach.

Use a good text editor that can open and search whole folders: examples of this would be Notepad++ or Sublime Text. With either of them, you can open the folder containing your target web app’s source code and search it all simultaneously for things like SQL queries, page names, functions, and other pieces of text to help you find vulnerabilities and piece together how the application flows. Both have or offer plugins with the ability to do things like hover over a function name in a file and get a link to other usages of it, the file it’s defined in, etc. For me personally, Sublime Text was probably the single most valuable tool I used for the course and the exam.

The exam guide notes that the search functions in some tools, like dnSpy and JD-GUI, aren’t great and could lead to you missing things, so don’t rely on them. You could use an IDE if you really want, like Visual Studio Code, but I find text editors less distracting and better suited for searching, with stuff like regex support. Hell, if you’re really hardcore, you could just use grep or vim, I guess. I do tend to just use vim for writing actual exploit code.

Save all your scripts from AWAE and reuse them for your exam: when I went through AWAE, I would make separate script files for each exercise in a module, even though they mostly just build on one another, just so I could have that progression documented. It’s also good insurance if you change stuff too much from one exercise to the next and need to go back to a last known good script.

They saved me a ton of time on the exam too, because I could just reuse the snippets I’d already wrote and not have to reinvent the wheel when it came to some complex nested loop for enumerating this or that. Even though you have two straight days to complete it in, your time is still precious and the less time you have to spend writing and debugging your own exploit code, the better.

When I did the course, all of the code examples were in Python 2. Of course, that version has been officially retired as of 2020 and everyone has been urged to migrate to Python 3. For the sake of simplicity, and not having to waste time porting scripts, I just stuck with Python 2 for the course and the exam. I’m not sure what this is all going to look like for the course going forward. The Kali Linux team has already made a statement regarding Python 2’s end-of-life and how it’ll affect the Kali distro, so hopefully the course material will be updated and examples ported to Python 3 soon.

Get comfy with debugging, databases, and reading logs: reading the code is one thing, seeing how it actually operates is another. I find that set breakpoints, stepping through the code’s execution, and seeing how variable values change with a debugger is invaluable when trying to break down complex code, especially when it comes to stuff like encryption functions. I would advise any AWAE student to get comfortable using Visual Studio Code for debugging, as it’s cross-platform, supports multiple languages, and has tons of helpful plugins. It’s very easy to debug an interpreted language, like PHP or JavaScript, on the web server with VS Code installed. Also make sure you understand how to use decompiled code and attach to a running Java or .NET application in order to debug it in realtime (hint hint, I wasted some time struggling with that on my exam).

Also, understand how to use logs to your advantage, especially database logging. You’ll cover it in depth in the course, and it’s the key to successfully figuring out a payload for something like a blind SQL injection vulnerability. Turn on logging in the database engine so you can see what your injected queries are looking like on the other end, which part of the overall query they’re being inserted into, what characters are getting filtered, etc. Also, don’t be afraid to use the database’s command line tool to play with the DB directly and test out your queries. I personally like to make sure I have a working query there before I start complicating it by having to replace filtered characters with alternatives, nest it within something else, or any other transformations that might introduce other errors into the mix.

Get comfy figuring out SQL injection payloads by hand: you’re not allowed to use sqlmap on the exam, so make sure you understand all the various ways you can evade filters, substitute different characters for others (like comments instead of spaces, double dollar signs for single quotes, etc.), pick out substrings, use logical operators, etc.

As always, RTFM: in this case, I mean read the exam guide carefully and make sure you’re doing everything you’re supposed to. You wouldn’t want to make it all the way through the exam only to find you missed some important step you can no longer go back and complete.

It’s a marathon, not a sprint: I’ve seen several other folks say this about the exam, and I’ll reiterate it. You might’ve gotten away with plowing through the OSCP exam with no sleep (but I hope not); you cannot attack this one full-on for two days straight. You’ll go nuts, exhaust yourself, and start hitting a point of diminishing returns. Take breaks, get enough sleep, eat meals, and clear your head out from time to time. If one of the challenges is driving you nuts, go look at another challenge.

It’s OK if you it takes you multiple attempts: there’s a reason why OffSec’s training and certificates are much more revered among pentesters than a lot of other infosec certs. Their exams are hard! I had to take OSCP twice and I feel like I was kinda lucky that I got OSWE on my first try. Of all the folks I know who’ve taken OffSec’s various exams, I’ve known brilliant people who got it on the first try and I’ve known brilliant people who’ve had to take the exams four or five times before they got it. Sometimes it’s just the luck of what challenges you get on your attempt and whether they line up with your knowledge and skills. It’s OK to fail, as long as you learn from it and keep trying. OffSec’s motto is, after all, “Try Harder!”

Good luck on the course and your exam!

My Grand Tour of Pentest Interviews

Late last year, I began looking for a new job.  Earlier this year, I finally got one!  I was interested in branching out into the broader world of penetration testing and red teaming, with more external clients and more broadly-scoped sorts of engagements.  This was something of a sell to prospective employers though.  I do have close to a decade of infosec experience, but only a few years of that is pentesting and I’ve always been an in-house pentester doing mostly web app and mobile stuff.  That means that I am something of a noob when it comes to breaking in from the outside; I’m familiar with a lot of the tech and methodology, just haven’t done a lot of it hands-on (outside of CTFs and stuff like that).  I’ve been in the broader industry for a while, meaning my salary requirements are a little higher, and I absolutely wasn’t going to relocate again so soon after my last move for my old job.

All of this and extremely high demand for pentesters at the moment meant I went through A LOT of interviews.  Some of them broke down over salary expectations.  Some of them I quit early because I could tell it wasn’t what I was looking for.  Some of them weren’t budging on relocation.  One I completely hosed myself on because I bluffed too hard during the salary negotiation phase.  At least one of them probably thought I was a complete dumbass.  But in the end, one employer won out and I’m now happily hacking clients, mostly from the comfort of my own home.

Besides having lots of experience being interviewed for pentest jobs, I also have some experience in interviewing people for pentest jobs.  At one of my previous employer, I was involved in telephone screenings and in-person interviews of a dozen or so different candidates to join our team there.

Because I went through so many different interviews recently and have experience trying to assess pentest candidates, I figured that put me in a unique position to grade these different companies and throw in my own opinion on the best way to do it.

My intent is half to just be amusing for those who are curious about how different companies are interviewing people, maybe those trying to find out what to expect the next time they start looking for a new gig; but I’m also writing this and hoping that some recruiters and hiring managers will see this.  I hope this will give you some insight into how your competition might be assessing candidates, what you’re doing right in your own process, what you’re doing wrong, and how you could be doing it better. Continue reading

Running KeepNote on a Mac

During my PWK training, I absolutely fell in love with KeepNote.  I used it extensively for tracking all the different networks, all the hosts in that network, all the different scan results and loot I’d collect on each, and also general notes about attack vectors I had tried, what worked, what didn’t work, what to explore, little code snippets or Linux commands for easy copy-and-paste use later, links to helpful articles…you get the picture.

Unfortunately, KeepNote isn’t very well-maintained.  As of this Fourth of July, the latest versions uploaded to the main site are five years old.  The original developer did put it on Github two years ago, but there haven’t been many pull requests accepted since then and he obviously doesn’t have the time to keep up with it anymore.  Totally understandable, but it sucks because I haven’t found a good, well-maintained replacement for it.

And believe me, I’ve tried!  Evernote, Bear, nvALT, desktop wikis like Zim, the macOS Notes app, and on and on.  No one else is doing, or even offering an option to do, the same kind of UI layout.  I don’t know why, but the three-pane arrangement of KeepNote just hits the spot for me.  The outline on the left side, submenu on top right, and notes section on bottom left; no one else arranges their app like this.

2017-07-01_11-50-59

For real, this is my happy place.

Continue reading