Hello Open Hash API

Introduction

Hash cracking plays a valuable role when penetration testing, as it is a great way to test technical controls and the human element in security. By leveraging known metadata extracted from plaintext passwords, attackers can achieve much higher success rates at recovering secret material from hashes and turn those into offensive capabilities.

Today, we want to announce a new open-source tool release, The Open Hash API or OHA system. The project has been in development and internal use for some time and could benefit the security community at large with an open-sourced version.

The Open Hash API system features two main components: an API server and an API client. The primary system's goal is to make it more accessible for security practitioners to maintain and develop better hash-cracking resources based on trends identified in data with a more custom approach.

The server acts as a remote data store and stores found hash material while being able to query, create, and maintain large amounts of hash data to give security teams the edge. Using the Gin framework, the server is fully containerized, configurable, and written in Golang. The client also provides an easy-to-use method to interact with OHA instances securely and seamlessly.

The public repositories can be found here:

• Open Hash API Server (OpenHashAPI-Server)
• Open Hash API Client (OpenHashAPI-Client)

We also have a publicly hosted version for the community to use and experiment with. Its hosted here at the URL below:

• Public Hosted Version (https://www.openhashapi.com/)

The server accepts data in the form of HASH:PLAIN and supports salted algorithms. Teams can use the server to load large amounts of hash data into a data format with a high degree of control over the storage requirements. This can allow teams to preserve valuable breach data and transform it into new offensive security resources.

Drop The Hammer On Hashes

With the application containerized, installation is a breeze, with the only requirements being to set up the backend MySQL database, customize your configuration file, and build then deploy your server image.

The application is created with the security community in mind. It offers several methods to ensure security, including HTTPS communication, Attribute-Based Access Control (ABAC) and user management, user registration, JWT authentication, request logging, and container segmentation.

Once set up, Open Hash API offers various unique features to enhance the internal offensive security team's workflows by using a large amount of information extrapolated from breached and cracked passwords to make data-driven decisions about approaching workflows. The tool also provides a collaboration platform to store breach data, create and download rules, wordlists, and masks, as well as an ephemeral platform for hash cracking collaboration. 

Why Store Hash Data?

Real hash data is the golden standard for credential security research, as humans often recreate similar patterns, keywords, and masks across environments and regions. This data can give hash-crackers the edge when approaching targets by implementing this metadata into their attacks to increase the odds of successful recovery when examining the strength of human-generated credentials.

For example, common trends and formats tend to permeate though human passwords when created under similar criteria and conditions. This makes it more likely for people to create similar passwords under the same policy and conditions, giving an advantage when attempting to reveal these secrets.

Uploading and Searching

OHA features a RESTful API that can allow for the uploading and searching of hashes. Many features have been implemented to ensure quality and give teams flexibility in storing and maintaining their data.

• Quality Control Filtering
• Automatic Upload Rehashing

When files are uploaded via the API, additional options are available for processing. The configuration file can provide a customizable regex to remove matched items. This option will automatically filter poor plaintext candidates such as, email addresses, poor-quality hashes, known junk values, and more before they even enter the database.

The API manages multiple hash types and supports a rehashing option, allowing submitted hashes to be rehashed into non-salted algorithms, which can help teams organize their data, reduce storage requirements, and control how things like frequency analysis are implemented. These features are optional, and teams can store material in their original formats.

Searching can be done by plaintext or hash value, and files can be uploaded and searched via the API. The server also supports the ability to have files uploaded that are disconnected from the database to better control data persistence. 

Data Security Considerations

One of the goals with the Open Hash API was to provide flexibility and security when approaching sensitive data such as authentication material. We realized that teams are often uneasy about storing sensitive data like hashes, and we saw that similar tooling generally always defaulted to database storage. 

We decided to offer two methods of data storage: one that would store data in the database to persist and another that would act ephemerally by storing files on the container's file system, ready to be deleted when the container gets torn down. 

Offering multiple methods to store data enables teams to choose how they want to handle sensitive data and ensure additional precautions are taken instead of simply relying on default logic. With this design in mind, teams can spin up instances at need and then control how and what data is stored. They can then tear down the instance without fear of data loss, as the backend database will persist on the underlying host.

Async Validation and Advanced Processing

If the server is set to rehash discovered passwords, an additional process can be implemented to validate hashes asynchronously. This process ensures that hash values are correct and accurate. This option provides teams with quality controls to make ingesting breach data a breeze without fear of poor quality data.

The server can reuse data to create new resources such as masks, wordlists, and rules. All stored items, including the validation feature, are automatically processed for multibyte and $HEX[...] values when transformed.

Generating New Wordlists and Rules

Open Hash API can use the stored resources to create wordlists, rules, and masks. These processes are kicked off asynchronously if supplied in the configuration file. These files can be accessed remotely via the API and are sorted by frequency of occurrence. This feature allows teams to flexibly fetch resources that meet their use cases and ensures they always use the most likely data.

The resources also offer a few options to help select the best type of content for the need, including choosing a certain number of records, determining an offset to start querying, filters for different types of rules. They also offer a way to search for only items that contain substrings.

Power Parsing Plaintext For Rules and Wordlists

One powerful feature of the system is the ability to parse down existing plaintext to create new rules and wordlists from hash material. These functions are designed to take plaintext material and parse password creation schema. This will extract portions of plaintext and transform them into rules and wordlists that can then be sorted by overall frequency.

This tactic ensures that generated material is always following patterns and emulating the target correctly. The backend database can also be customized depending on teams needs to target specific patterns and metadata. 

Use Cases and Looking Forward

We are excited to announce the release of a community version of the Open Hash API and continuing development with the community in mind. We aim to include many more features to make OHA a valuable tool ecosystem for offensive security practitioners.

The tool was designed to be flexible and support several use cases depending on the need. Teams will find themselves gravitating to certain features depending on their hash cracking workflows, and we wanted to provide an option to support those needs.

Let's start with a small wordlist of likely metadata tokens and grab rules that will extend the candidate by appending likely data. The append=true filter we are applying will ensure candidates contain append rules.

Then, in a fabricated example, let's run the candidates against our captured hashes:

Another good use case is for improving attacks when key data is known, such as the password policy or seeking target rules depending on the attack type. For example, if a team was using a base candidate wordlist or reusing breached passwords, certain rules would be excluded or included because of the data format.

In this example, we will grab a wordlist from Open Hash API and look for proper rules for our target. We can assume that our target will require a complex password needing one uppercase, lowercase, digit, and special character. Because our wordlist is only lowercase base words, we would likely want toggle rules to ensure complexity.

We can take this to the next level as well and look for increasingly complex rules by including or excluding criteria. The rules are always sorted by frequency, so no matter the filter you can find results you are looking for.

The tool may still have a few bugs and improvements to be made, but we hope the tool is helpful in the following scenarios:

Super Potfile for Hash Cracking

• Open Hash API provides the ability to store plaintext and hash material in a data-centric manner with the ability to enforce a proper authorization model surrounding access, which can be used to empower teams in making data-centric decisions. With the ability to rehash items, a single algorithm can be maintained for optimized storage or multiple can be stored.

Rules, Wordlist, & Metadata Database

• With the ability to reclaim metadata from entries, the server can return value to security teams by generating likely wordlist, rules, and masks based on the input data. With quality filtering and validation, items are ensured to be high quality.

Collaborative Framework for Hash Cracking

• The ability to upload files enables teams to act together when they encounter hashes without uploading them to the central database. The uploaded files can be easily updated when valid plaintexts are found for any matching hashes. Teams can also use this to pass around temporary files with authentication.