Rectify.IT – Arachni Web Application Security Scanner Framework on Kali/Ubuntu

Did I mention that I was doing some study in Cyber Security? Over the past couple of years, I made a decision to do something about it, and eventually got certified in a number of certifications. It all started with CCNA CyberSecurity Operations – a free course that came up in NSW Tafe. When I heard that a friend was going for it, I decided to join in, especially as it was free at the time (it isn’t now). Then afterwards, I decided to get trained in CompTIA Security+, followed by CompTIA CySA+ and then naturally continued with CompTIA PenTest+. I was able to get through the exams ok, and actually thought that was going to be enough, but then the opportunity came around last year to do EC-Council’s Certified Ethical Hacker.

CEH is something that I had been wanting to do for many years, and finally last Saturday, I sat for the CEH exam and to my surprise, passed. We had been training in CEHv10, but this year, the exam went to CEHv11 which introduced additional concepts and had a very much hands on approach, so there were a lot of questions about the actual tools that hackers would use to perform their nefarious activities. A few days ago, I received my confirmation that I am now a Certified Ethical Hacker.

Ok, I might be digressing – but during CCNA CyberOps, we looked at web application security and Arachni was one of those popular applications that could test websites for vulnerabilities. It wasn’t the only one, but it had a large following. We also looked at this during the CompTIA training especially in CySA+ where we want to know what vulnerabilities our web applications have, so that we can fix them or mitigate them.

Late last year, my teacher mentioned that he had some problems running Arachni on Kali Linux. I did remember that I had tried this some time ago, and had mixed results. Yesterday, I looked up my notes on Arachni, and it seemed straight forward, download Arachni, unzip it, start the web interface, then log on to http://localhost:9292/

I did this and much to my surprise, when I put in the default username and password to logon to the web interface, I get a screen showing the following:

Naturally I went to Google for a solution, as we usually do, but the solution was not forthcoming. There were various fixes that people had suggested, but the comments were that it might work for someone, but not for others. I found people having this problem from a year ago, and it seems that in January 2020, the development of this application was stopped, which meant that support was no longer available.

I did have an older Ubuntu virtual machine, version 19.04 actually, and I installed Arachni on it, and it was working fine, so the problem seemed to be related to newer releases of the operating systems. Ubuntu 19.04 though, was no longer supported, and could no longer update patches. Then I downloaded Ubuntu 20.04 Desktop LTS. I installed this in a new virtual machine, installed Arachni and tried to log onto the web interface, and got that “We’re sorry” message just like almost everyone else is getting. I could access the log file, and it showed that there was an invalid hash during the logon process – that’s interesting. Sure enough, this is the same problem that others had been reporting as well, for a year or so, and no concrete solution was at hand.

This morning, I was up early and thought I should try to get to the bottom of this. There was a suggestion by someone that it was the database – it might be corrupted or missing. Since I knew that Arachni it was using a sqlite3 database, I was able to download a Sqlite3 database browser, and extracted the production.sqlite3 database file from my machine (by the way, the database did exist! and hacking is what we do!)

I opened the database and browsed to the users table and could see the encrypted password, which is a bcrypt hash. I copied each hash, then tested it in an online bcrypt generator site which allows me to check password hashes against a known password. Both of those hashes were fine, when I entered the expected password. So these tests validate that the database is accessible, intact and not corrupted.

Next step, is to narrow down the problem. I ran the command to change the password for a non-existent user. This was ./arachni_web_change_password user1@user.user 12345

Good, I get a message saying that the user email address was not found – which was what I expected, but wasn’t sure that I would get that. Ok, how about I create a new user account – ./arachni_web_create_user test@test.test 12345 tester (had to add the full-name) – and I got the invalid hash message as seen below:

Now we are getting somewhere by ruling out possible causes. I went through the password.rb Ruby file and the error comes up because the password hash passed to it did not pass the valid_hash test.

Ok, so the hash wasn’t valid – what was the hash that it was trying to test? I then modified the error line to replace “invalid hash” with raw_hash, then tried creating a new user again to check the error log. I did this by ./arachni_web_create_user test@test.test 12345 tester

We are making progress! I can see the bcrypt hash, can you? I know that it starts with $2a$10$… so this is bcrypt with 2*10 rounds and should be followed by a salt and then the password hash, but what is that garbage on the end. I copied the readable parts of the bcrypt hash, then tested it and failed. I then tested it again, but this time I only used the first 60 characters of the raw hash and it passed the hash test with the 12345 password.

Great – we are getting somewhere! This means that something in the bcrypt process was giving extraneous characters, which is why the generated hash did not pass the valid hash test. After some further digging, I traced it going back to the engine.rb file – line 51, and decided to just try truncating the generated hash. This would not be a fix, but would be a good workaround, and a simple one at that. The change would be like this:

__bc_crypt was somehow returning a hash that had some corruption at the end, so the addition of [0…60] is saying that I only want the first 60 characters (truncating the returned hash). Why 60 characters? Let me explain – this bcrypt hash is comprised of, $2a$ which is the algorithm, then 10 is the cost parameter, i.e. 2 to the power of 10 key expansion rounds, a $, then the salt which is 22 characters long followed by a 31 character hash. If I got my maths right, it means that the bcrypt hash is 60 characters long. Having already tested the first 60 characters of the generated hash, I was very confident that truncating the hash would end up with a valid hash, and therefore – no more invalid hash error.

Time to try this out on one machine – by making that change to the engine.rb file. Fantastic, it worked, Eureka! I then continued with my other test machines. Does it continue to work? Yes, it worked on my Ubuntu 20.04 desktop. It worked on my Parrot OS virtual machine on my laptop. It worked on my Kali Linux 2020.02 virtual machine, it worked on my Kali Linux 2020.04 virtual machine. Now, I have to say that when I say that it worked, I mean that I was able to log in to the Web Interface – which was the main problem that people were having, you remember? The “We’re sorry, but something went wrong” problem.

This is not a fix, but a workaround. A fix would mean identifying the underlying root cause and then rectifying it. But a workaround is as good as a fix – get it? Now whether or not Arachni will be completely usable is unknown, especially as it is no longer being developed, that means that newer vulnerabilities cannot be included unless someone takes up this task.

My testing shows that the Arachni framework is still usable when implementing my workaround, but did note that when using Kali 2020.04, I could log on, but when I configure a scan, the scan didn’t seem to do anything and timed out after 10 minutes.

After it times out, I can see an error:

I think I might have to leave that problem for another day!

To give a summary of this problem – if you are trying to log into Arachni Web Interface, and are getting that cryptic message that something went wrong, try modifying the file located at the path system/gems/gems/bcrypt-3.1.11/lib/bcrypt/engine.rb, find line 51 and add [0…60] to the end of the __bc_crypt command – like this:

Then you need to stop the arachni_web (if it is running) and then start arachni_web, and you should be able to log on without that message. One more thing, I forgot to mention that Arachni also works on Windows 10 – I tested and confirmed that yesterday and I didn’t need to do anything to it.

[P.S. I did run some scans of my web servers from those machines using Arachni. I even pointed it at a Metasploitable2 machine, which is a known vulnerable machine. The scans worked, with the exception of course, of Kali 2020.4 – but you knew that already!]

2 thoughts on “Rectify.IT – Arachni Web Application Security Scanner Framework on Kali/Ubuntu

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s