Are You Really receiving a Penetration Test Report?

There are more and more organizations interested in a penetration test, or simply a “pentest”, on their infrastructure. However, there is a requirement for specific skills and this expertise is not often available within most organizations. It is also a good idea to have an external opinion, someone who will be impartial and doesn’t know too much about the current configuration. Thus, it is often necessary to hire a security firm to accomplish this task. Unfortunately, there are still many security professionals who are not well qualified to provide this service. It could be intentionally or not. But many clients are easily fooled by consultants, even more when it is a question of IT security mandates. Clients need to be careful and be able to recognize what the report should look like.

Vulnerability Scan != Penetration Test

There are many phases performed during a penetration test. I will describe these phases in more details in an upcoming post. However, it is important to know that there is always an active recognition phase where the professional will use automated tools. For example, Nmap, Nessus, OpenVAS, Nexpose, etc. These tools will allow to scan one or more IP addresses for open ports and well-known vulnerabilities. There are often false positive items in these reports and the professional should validate these findings. For example, a Nessus report and Nmap output. This is an important phase to help the professional to gather information about the target.

The problem is the fact that some security consultant will sell these reports as penetration test. To run vulnerability scans can be a valid service offered by a consultant who will filter findings and rank them according to the exposed risks. However, it cannot be introduced as a penetration test report.

A complete penetration test report will include many other phases. After the previous phase, the security professional will manually look for other vulnerabilities with different methods without the use of automated tools. Finally, the main objective is to retrieve confidential information from the target that someone should not have access and to gain a remote access to the target. To achieve that, the security professional will actually try to exploit vulnerabilities previously found.

Things to look for


It is not possible to perform a well-done penetration test in only a few hours. Someone told me recently about a consulting firm who consultants will perform two or more penetration test reports during the same day. It is impossible to have great results in these conditions. Obviously, it is often a question of cost. So it is important to validate the duration of a penetration test. It should be in days but always depends also on the scope.


If there is only one target with no public service running on it, there is not so much someone can do. But probably also worthless to have a penetration test in this context. The line can be thin between a too limited scope where it will not represent what an attacker would have access, and a larger scope where an attacker will lose time on low risk targets. Should it be performed from the Internet or from the internal network? On the other side, if you receive a penetration test report from a third-party vendor, be sure to validate that the included infrastructure and applications are the one that supports the received services.


Even a professional with an impressive background in information security is not necessarily the best pentester. I often see job positions related to “ethical hacking” where a certification like the CISSP or even the CISA would be a requirement. No, I don’t have words for that, just, no. If you are really looking for a certification, the best in this field would be the OSCP. But even then, I have it, and I would not recommend myself to perform a penetration test. Why? I did the exam and never practiced anything related to that field afterward. Someone who is passionate about this field and compete at different CTF will be the best one for this task.

Update: CISA Certification and Frequently Asked Questions

CISA logoIn August 2014, I published a post about my experience with the CISA exam and the required experience. Even 3 years later, it is still the most popular post here and not so long ago, I was always seeing more requests after the exam dates. However, it seems that exams are not on specific dates anymore but within 3 specific testing windows throughout the year. I am still surprised by the number of comments I received on this post and I wanted to do an update with some recurrent questions.

Exam Before the Required Experience

It is absolutely possible to pass the exam before having the required experience. I would even recommend it to show your interest toward this field. Your current or future employer will recognize the effort invested in passing the exam. However, you have to keep in mind that you will have to obtain the required work experience and send your certification application within 5 years from the date of the exam. If you really need 5 years of experience and it is not possible to have any waiver, maybe the timeframe is unfortunately too short. But if you only need 2 to 3 years of experience, it is a good plan, in my opinion.

Even if you can’t introduce yourself as being CISA certified, it will definitely be an advantage to mention during an interview that you passed the exam. You will also receive a letter from ISACA with your passing score that you can use as a proof.

Substitutions and Waivers

You will need a minimum of 5 years of work experience in order to obtain the certification. It goes without saying that you need to pass the exam too with a score of 450 or higher. However, it is possible to obtain 1 to 3 years as substitutions and waivers of this experience. In any case, you will always need at least 2 years of work experience related to CISA domains.

  • One year : Any work experience in information technology (IT) or any work related to auditing. For example, financial auditing or compliance work experience such as SOX would be a valid auditing experience.
  • One year : For 60 completed university credit hours which are normally 2 years as a full-time student. The credit hours could be for an undergraduate and/or graduate degree. There is also no mention that you actually need to have obtained the related degree, as long as you have proof that you have completed the courses. It could be from any recognized university, on-site or distant learning.
  • Two years : For 120 completed university credit hours which are normally 4 years as a full-time student.
  • One year : With a master in information security or information technology.
  • One year : With a bachelor or a master degree that enforces the ISACA-sponsored Model Curricula.


The easiest way to obtain the work experience for the CISA certification is obviously to be an IT auditor. But it is not always possible for everyone and it is definitely not a requirement. It is important to carefully read all the job practice areas including the task and knowledge statements. There are many candidates who are surprised by these statements. The CISA practice areas include many work fields in IT and not only IT audits.

CISA Application

You will have to get the work experience 10 years before the certification application or within 5 years from the date of the exam. You only have to send your application to ISACA when you actually have all required experience. In any case, you absolutely have to submit your application within 5 years from the passing date of the exam.

Financial to IT Audit

If you are already performing financial audit with a CPA designation, it is absolutely possible to do the transition to IT audits. You could simply ask to be more involved with IT controls. You will already have the advantage to have an understanding on how audit works. There are many IT General Controls (ITGC) that don’t require advanced IT knowledge in order to be adequately audited. The Big 4 use an army of interns to audit these IT controls.


There are many more jobs related to the CISA than being an IT auditor. Many positions related to IT risk and compliance, even information security. However, there are other certifications if you want to work in information security and a candidate with the CISA would not be my first choice. I would recommend doing a search on a job posting website such as Indeed.

Worldwide Recognition

ISACA and the CISA certification are well recognized worldwide. A quick search on a job posting website and you will probably have many results. Furthermore, there are more than 200 local chapters worldwide.


When you are officially awarded with the CISA certification, you will then have to complete your continuing professional education (CPE) credits. The ISACA membership is not a requirement. However, I would recommend it to have access to many free resources in order to obtain your CPE credits.

ISACA Website

All information provided here could change anytime and the reference must always be the ISACA website. The page “How to Become CISA Certified” is mainly the reference about common questions related to the CISA certification. If you are unsure about your experience and possible waivers, I would recommend validating directly with ISACA and/or your local chapters. Each application is different and they are the only one able to validate your application.

Cloud Security with Object Storage

AWS logoMany cloud providers are often criticized for the security provided with object storage services. Even more after the disclosure of private information that occurred in 2017 by using these services. These security breaches were also from well-known organizations such as Verizon, Accenture, Booz Allen Hamilton, Viacom, National Security Agency, National Credit FederationAustralian Broadcasting Corporation, Department of DefenseRepublican National Committee, etc. There are often new organizations to add on this list but they are the main one from the last months. These organizations were mainly using the object storage service S3 from AWS.

Object Storage

This is not a technology only provided by AWS with the S3 service. There are many services provided by other well-recognized cloud providers to store files in the cloud such as Azure, Google Cloud, DigitalOcean, IBM, etc. However, AWS S3 is definitely the object storage service that is the most used by many organizations. The service was also first released in 2006 before other services from competitors. The statistics are a little bit old but as of April 2013, AWS mentioned that S3 has more than 2 trillion objects stored with 1.1 million requests per second. In 2018, it is possible to assume these numbers are even higher.

Amazon S3 is often wrongly targeted by the media. It is simply the most popular service used by many organizations of all size. We have to keep in mind that object storage is only a way to store files, often with a cloud provider but it could also be with a private infrastructure.

IT Administrators

I often read some IT professionals and even information security professionals to have doubts on these services. Mainly doubts on the security measures available to protect the information stored. It is important to understand that security breaches related to object storage are often not related to the underlying technologies. Cloud providers such as AWS, Google and Azure are able to provide secure environment for your files. The configuration for such spaces, or buckets in S3 terms, is secure and private by default. How is it possible in this case to have public files on the Internet?

Simply ask your IT administrators. It is more a question of misconfiguration. In order to authorize a public access to the stored files, someone actually needs to perform a manual action to change the default behaviour. The approach would be different for each service but the principle is the same. It is possible to manage accesses on S3 with rules but other services could be simpler with an option to be set at “Private” or “Public”. This is often a configuration available for the space and/or per file.

Maybe it is the time to review the accesses implemented for your files stored in the cloud? From object storage services like S3 but also on services like Office 365, Dropbox, Google Drive, etc. It is so easy to forget about a file that should not be available for all on Internet.

Third-Party Vendors

Are you aware of your third-party vendors who could use object storage with your information? For example, with Verizon and Republican National Committee, in both situations, third-party vendors were involved i.e. Nice Systems and Deep Root Analytics. Organizations easily trust more and more third-party vendors and share confidential information. This data can be about the organization operations but often on clients. Nevertheless, it is important to evaluate the information sent to external vendors and to understand how this one uses the data.

NIST and the Digital Identity Guidelines

The NIST published last June the final version of the Digital Identity Guidelines also known as SP 800-63. This publication was a draft since 2016 and they even asked for comments from the community on GitHub during the summer 2016. All these comments were inputs for the final publication. Many posts on the Internet mention these changes. But I think it is still important to reiterate them since they are not necessarily well known by everyone who is not in information security.

Who is the NIST?

The National Institute of Standards and Technology (NIST) is a non-regulatory government agency of the US Department of Commerce responsible, among other things, to publish standards for federal agencies. The Special Publications (SP) 800 series are well known to be important guidelines in the information security field for private and public organizations. Worldwide professionals value these publications and they are often used to structure their information security strategies.

New Requirements

Since the past two decades and more, we all saw the result of these requirements. A lack of user experience where most users were often able to circumvent the rules. Many studies have demonstrated these requirements were adding little value on the security side. Furthermore, users were often able to find a way around these requirements thus reducing the security goals. There are mainly 3 requirements updated:

Password complexity rules

You know the rule where you have to put at least a lowercase character, an uppercase character, a number and a special character? It’s not a requirement anymore. Studies shown that users were simply using different patterns to respect this requirement. For example, one trick was to replace some letters by numbers or even simpler, to add an exclamation point at the end. These patterns are all well known by hackers and these passwords were not more secure because of the complexity rule. Oh, all characters should also be available, even emojis!

Password Expiration

This is even something audited during common external audits. That moment at work when you receive a notification and you have to change your password. And this, often every 90 days. No more! We all know users were keeping the same password and adding a character at the end. I was the first one to do it because I always thought it was not efficient. It’s better to have one good password for the service than having a weak one changed every X days. However, it could still be possible to force a user to change a password in certain situations. For example, it should be possible to request users to reset their passwords if the service suspect a compromise. So, it is still important to keep a password history.

Password Hints and Knowledge-Based Questions

Financial services are really good with this requirement, mainly for the knowledge-based questions. This is when questions are also asked with a password to complete the authentication process. The main problem with this is the fact that most answers are now freely available online with social media. For the password hint, I never understood this one. I always saw this one as “let’s just give more clues to hackers on my password” so I never used this one.


The next two are interesting recommendations from the NIST:

Common passwords and usernames

With the new requirements, users are not forced to choose specific characters or to change password. However, the NIST suggests that passwords should be validated against a dictionary of well-known passwords and/or a list compromised passwords. This recommendation makes so much sense. For example, if a user is trying to use, “Test123!” would hopefully fail the validation against the dictionary. There are many dictionaries available online. The logic behind that is the fact that hackers are using these dictionaries to find passwords. The situation is also applicable to common usernames such as “admin” or “root”.

Multi-Factor Authentication (MFA) With SMS

At one point, the NIST completely removed SMS as a valid method to implement with the multi-factor authentication. But with the final release, SMS is still supported, but not necessarily recommended. It is theoretically possible to intercept an SMS. However, it is still more secure to implement MFA with SMS than having no MFA at all. The alternative would be to have an app such as Google Authenticator or a solution with push notifications such as Duo.

October 2017 : Security Breaches

The data security breaches occurred/disclosed in October 2017.


DISQUSThe popular commenting system was breached in 2012. Disqus got notified by Troy Hunt, a security expert, who obtained a copy of the data. According to the company, the data exposed are from 2007 and involve 17.5 million users. Among the user’s information stolen include email addresses, usernames, sign-up dates and last logins. However, about one third or approximately 5.8 million users, also got their passwords in the wild. At least, the passwords were not in clear text but hashed with a salt with the now weak SHA-1 algorithm. They seem to have handled the situation well with a public disclosure in 24 hours and they asked the affected users to reset their password account. They have also mentioned that they are now using the bcrypt algorithm which is now the best practice.

Far Eastern International Bank

Far Eastern International BankA malware infected this Taiwanese bank which instructed the SWIFT terminal to move $60 million into different bank accounts based in Sri Lanka, Cambodia and the United States. SWIFT is the main global banking network where it is possible for banks to exchange funds between them. It is not the first time this situation occurs and a well-know breach occurred in 2016 with a Bangladesk bank where the attempt was to steal $951 million. The Far Eastern International Bank was able to retrieve most funds. Mostly since the breach in 2016, the SWIFT organization has developed a more stringent security requirements for their customers with the Customer Security Programme (CSP) but many banks are still in the process of getting certified.


AccentureThis is another big name in the IT consulting industries. Accenture offers consulting services for the largest organizations and often seen as a leader in cloud consulting services. UpGuard reported that AWS S3 buckets were configured for public access. In total, 4 buckets were available for everyone. These buckets contained confidential API data, customer information, private keys, 40 000 passwords mainly in clear text and even logs from a monitoring solution. One bucket contained more than 137 gigabytes of data.


Remember the data breach that occurred in 2013 at Yahoo? It was first disclosed by the company that someone had access to information on one billion accounts. This number was revised by Verizon, the now parent company of Yahoo, at 3 billion accounts. It was possible to retrieve the usual information such as names, email addresses and hashed passwords. Some hash would still be with the weak MD5 algorithm.

Hyatt Hotels

It was possible to obtain the information from cards manually entered or swiped at the front desk. This situation occurred between March 18, 2017, and July 2, 2017, in 41 properties across 11 countries. As expected, it was possible to get the cardholder name, card number, expiration date and verification code. This is the second security breach for this company.

Pizza Hut

About 60 000 customers might have been impacted by a security breach that would have occurred from the morning October 1, 2017, to midday October 2, 2017. Data including customer names, billing postal code, delivery addresses, email addresses, and payment card information. Pizza Hut notified by email customers impacted only 2 weeks after the situation and they are offering a free credit monitoring service for a year.

South Africa

66 million records were obtained on South African. What, wait, the population is only about 56 million people? The obtained database also included 9 million people with a deceased status. The database was openly available on a web server owned by Jigsaw Holdings and was probably bought from a credit bureau in 2014. Information available include South African ID number, name, gender, age, location, marital status, estimated income, address, phone numbers, employers, etc.

Patient Home Monitoring Corporation

An estimated 150,000 American patient files were available through an unsecured AWS S3 bucket. It is hard to know for how long this bucket was available with public access but Kromtech Security Researchers have discovered the breach on September 29, 2017. 47.5 gigabytes of data with about 316,000 PDF files including mainly blood test results. These documents contained names, addresses, contact information, dates of birth, diagnoses and names of physicians. All this information is strictly regulated by the Health Insurance Portability and Accountability Act (HIPAA).


When you are a major company who is developing software and hardware, you have a central database somewhere to track and document all vulnerabilities related to your products. Of course, this database contains critical information about your products and you probably prefer to keep it secret. Well, this database at Microsoft was hacked in 2013.