Historical Data Sets

Introduction

Empirasign's primary focus is bringing comprehensive and correct real-time Fixed Income market data to our client base. Our web application and REST API have been the two basic modes to receive these data. Today we introduce a new data offering, Historical Data Sets, for those whose use cases are not tied to real-time.

Available Historical Data Sets

BWICs

Since 2010, Empirasign has been the market leader in capturing all Structured Products BWICs and related data (price talk, covers, re-offers, etc.). Our broad-based data contributor network has allowed us to set the standard in this arena. This data set comprises price talk and trade color for approximately 350K BWICs.

Runs / Indications of Interest / Axes

• Empirasign has been capturing dealer runs (bids, offers, and markets) in Structured Products since 2011--over 15 years of history.
• Our Corporate Credit (IG / HY) data set goes back 3+ years with nearly 2 million unique quotes per day.
• Our Levered Loans quotes data set goes back 2 years.

Holdings Data

Since 2020, the SEC has mandated Mutual Funds and ETFs report month-end positions and valuations for all holdings. We've cleaned and normalized over 300K filings to create this data set which now comprises nearly 50MM records across 700K unique CUSIPs and ISINs. This data set covers all sub-sectors of Fixed Income.

Deal Documents

Over the last few years, we’ve built an unparalleled Deal Documents library. Nearing 20,000 documents, these include Offering Memoranda, Prospectuses (and Supplements), Term Sheets, and Indentures. In addition to source materials, data are available pre-chunked and vectorized ready to serve as inputs to your machine learning models or in-house RAG systems.

Data Formats

• CSV
• JSON
• SQLite

Delivery Methods

• One-time Links: Download historical data directly from your browser via a one-time use link. This method is suitable for data extractions up to 10 MB in size.
• SFTP: We can push files to your SFTP server. This method is more robust to communications hiccups that can interrupt large data transmissions. We can push files up to 10 GB in size using this method.
• S3: S3 is the newish FTP. It's super redundant and super high bandwidth. We've made 100+ GB data deliveries to clients without incident using this method.

Example Use Cases

Back Testing & Algorithmic Trading: Quantitative analysts use historical prices to run simulations on trading algorithms to assess strategy performance. Given the breadth of these data sets, practitioners are now uniquely positioned to measure challenging phenomena such as slippage, liquidity patterns, and optimal execution algorithms.

Machine Learning & AI Model Training: Data Science Engineers need massive, clean data sets of historical prices to train deep learning and machine learning predictive models.

Risk Management & Stress Testing: These data sets can be used to analyze and identify market crashes and anomalies (e.g., the GFC, the Taper Tantrum, or the 2020 market crash). Historical Data sets can be used to empirically calculate Value at Risk (VaR) and ensure portfolios can withstand similar historical shocks. Furthermore, data can be resampled to create empirically grounded stress test scenarios.

Quantitative Research & Academic Studies: Academics and researchers use long-term data sets to analyze market anomalies, study the efficiency of markets, and publish financial papers. Additionally, these data sets offer a plug and play solution for econometrics, machine learning, and statistics coursework.

INTC Stock Price vs Dealer Bond Quotes
An example of using Empirasign Historical Data to study prices across asset classes.

Conclusion

We introduced you our Historical Data Sets offering. If you would like to further discuss this new offering, or other matters, please reach out to us at [email protected] or 646-837-8848. If you prefer to explore these data sets before discussing with us, we encourage you to sign up for a Free Trial to our web application. Every data set detailed above (and more) are available via our Web App or REST API.

Dynamic Watch Lists

Introduction to Empirasign Watch Lists

Since the earliest days of Empirasign, clients have had access to our Watch List feature. Copy and Paste a list of bonds into a form, and we'll alert you via a pop-up and email whenever any bond in your Watch List appears on a BWIC or dealer run. In addition to real-time alerting, clients also receive end of day, week, and month recaps in Excel. If you're not yet using this feature, get started by heading over to the Create Watchlists list page.

Making Watch Lists Dynamic

A few years ago we added the ability for clients to create deal and issuer-based watch lists. This feature can be found here, and we have a short write-up on our blog as well. Today, we are making issuer-based watch lists dynamic. That is, once an issuer-based watch list is created we'll automatically add new bonds to this list as the issuer brings additional bonds to market.

To Use This Feature

• Head over to the Create Deal Watchlist page.
• Enter an issuer's ticker (AQFIT has been keyed in the example screenshot below).
• Click on the Select All option in the dropdown (circled in red in the screenshot).
• The complete list of bonds will then populate on the right-hand side. Click on the "Add selected tranches..." button as shown in the second screenshot to create your Issuer-Based Dynamic Watch List.

That's it. Issuer-based Dynamic Watch Lists have a "set it and forget it" simplicity not seen in the market since the heyday of countertop rotisserie chicken.

Screenshot of Create Deal Watchlist page: www.empirasign.com/deal-tracker/

The screenshot below shows all currently issued bonds for the Aqua Finance Issuer Trust (AQFIT). Clicking on the "Add selected tranches..." button (circled in red in the screenshot below) will create an Issuer-Based Dynamic Watch List.

Whenever a watch list is detected as single issuer, it will be marked as dynamic by our systems. This status is indicated at the top off the Manage Watchlists page when viewing such a list (the status line is circled in red in screenshot below). Multiple times per day, our systems will review all client Issuer-Based Dynamic Watch Lists and add any new bonds that have been brought to market.

Conclusion

We introduced you to our Issuer-Based Dynamic Watch List feature. We would like to thank TL who pointed out the need for this functionality. If you would like to further discuss this new feature, or other matters, please reach out to us at [email protected] or 646-837-8848. If you or a colleague would like a Free Trial to Empirasign, please head on over to our Free Trial Registration Page.

Python SDK on GitHub

Introduction

For over ten years, Empirasign has made its Market Data available via a REST API. Our Parser API has been publicly available since 2017. While any REST-based API is language agnostic, Python is the language of choice for most of our clients. To these ends, we have worked on building new and improved Python access classes, utility functions, and example scripts. Most of this code has been housed at a variety of links and pages across our site. Today, we announce the unification of all this code under one Python-based SDK hosted on GitHub.

NO BREAKING CHANGES!

This SDK and its new methods and scripts introduces no breaking changes for anyone using legacy API methods. Only new and good things are introduced herein.

What's in the Repo?

Market Data API

Version 1 of our Market Data API is much easier to work with than the legacy version. All access methods are encapsulated within one utility class. Authentication methods are now consistent across all API endpoints.

Parser API

Similar to the Market Data API, all access patterns and methods are handled via one utility class. This class allows for email message submission in either RFC or MSG format. In addition to parsing endpoints, utility endpoints which handle ticker and identifier resolution can also be found here.

The SDK also includes a number of example scripts for common client use cases.

bwic_poll.py: Make a real-time local copy of all available BWIC data (list contents, price talk, and trade color). In this example, SQLite is used for data persistence, but the code can easily be modified for any other SQL database. Once you create a local store of real-time market data it's a breeze to integrate this data with other internal systems. These can include cash-flow engines, CRM, risk, and valuation tools. There is another example script, runs_poll.py, which does the same thing, but for runs data.

eod_data_grab.py: If your market data needs are close-of-business driven, this is the most effective way to utilize the Empirasign API. This script is also useful to those who want to ensure they never miss a quote or BWIC. There is a longer write-up available here.

intex_mktd.py: Use the Market Data API to keep your INTEX market data directory well-fed. When this script is in action, INTEXCalc will automatically have access to Empirasign's latest market data. We've written more about Empirasign - INTEX integration on our blog, here and here.

parse_run.py: This script will parse a run email from the command line, and is great for testing. If you'd prefer a GUI for such tests, please head on over to our Email File Parser blog entry. When you're ready to industrialize this process seek out the parse_imap_inbox.py script.

Getting Started

1. pip install git+https://github.com/empirasign/python-api.git
2. Obtain Parser or Market Data API credentials from your Empirasign Representative.
3. Get started building!

Legacy Resources

All legacy resources will be maintained for the foreseeable future. GitHub is great for code hosting and providing updates and change logs, but our website will remain the best place for tutorials, data dictionaries, and other resources. A few pages you may want to keep in mind are linked below.

Legacy Market Data API Documentation

www.empirasign.com/api-mbs/

Developer Resources

www.empirasign.com/developer-resources/

Parser and Market Data API OpenAPI Specification (fka Swagger)

www.empirasign.com/v1/parser-api-docs/
www.empirasign.com/v1/market-api-docs/

Conclusion

We introduced our new Python SDK. We also highlighted a few example use cases and where to look for documentation. If you would like to further discuss these items, please reach out to us at [email protected] or 646-837-8848. If you or a colleague would like a Free Trial to Empirasign, please head on over to our Free Trial Registration Page.

Bond Valuation Work Using Empirasign

Introduction

Empirasign serves two core markets. We serve the Front Office doing the transactions and the Middle Office assigning fair value levels to these transactions. Due to transparency issues and the general complexity of bonds, valuation can be a particularly thorny problem in Fixed Income. As work begins to ensure the accuracy of quarter-end and year-end marks, now is a good to time to present an overview of the data sets and tools Empirasign offers to turn this thorny problem into a tractable one.

Independent and Bias-Free Market Data

Before we begin, it's important to note that Empirasign does not own bonds, nor do we make markets in bonds. We function as an independent observer of the market. As such, the Market Data acquired and published by Empirasign is free of bias and uniquely suited for valuation work.

Data Sets

Empirasign BWIC and Runs Data Sets

Since 2010, we've built up a global network of buy-side and sell-side Market Data contributors allowing us to give Empirasign clients an unparalleled look into the formerly opaque BWICs and Dealer Runs of the Structured Products Market. Over the last few years, we've also added Corporate Bonds and Bank Loans to our crowdsourced data sets. And since 2024, we've been indexing the New Issue market on a real-time basis. To fill out this mosaic of first-party contributed data, we've licensed third party data sets as well.

NAIC Data

Every insurance company in America must report all transactions to their self-regulatory body, the NAIC. We license these fixed income transactions, and display them to you. This data set comprises over 19MM transactions (including price, notional amount, date, and counterparty) across 670K unique ISINs (Structured Products, Corporate Bonds, Loans, and Private Credit). You can read more about this data set here: www.empirasign.com/blog/NAIC-Transaction-History/

N-PORT Data

Every US-based Mutual Fund and ETF must file this report on a monthly basis. The quarterlies are subsequently publicly released. In this report, each fund details every security holding and the price they marked it at for quarter-end. In sum total, there are over 43MM unique observations comprising over 635K unique ISINs across Structured Products, Corporate Bonds, Loans, and Private Credit. Given that these prices represent the primary input used to compute NAVs, the N-PORT data set is effectively the world's largest test bank and answer key. Back test your valuations against what others have reported, and look to this data set in times of market duress. Furthermore, we also create insightful reports leveraging this data set (more below). For more information on the N-PORT filing and resultant data sets published by Empirasign, please consult this blog post: www.empirasign.com/blog/NPORT-Filings/

TRACE Data

All BWICs are linked to TRACE data where possible, and we have TRACE data for all Structured Products on an end of day basis.

Deal Documents

When conducting a deep dive it helps to consult the docs. We have Offering Memoranda, Term Sheets, and Indentures available for 1,000s of deals and all are fully searchable.
www.empirasign.com/blog/Deal-Documents-Full-Text-Search/

Tools

Watchlists

With a simple copy / paste from Excel, Empirasign clients can create and maintain watchlists for 1,000s of securities. Once a watchlist is established, a number of reports are automatically delivered to you. Every day after the close, you'll receive an Excel-formatted report for all watchlist market activity. We also send one on Friday afternoon which recaps the week's activity. And most importantly, we send out one at month-end. The month-end report will save you untold hours of data gathering and cross-checking. For over ten years, these Excel recaps have been deeply embedded into our clients' workflows.
www.empirasign.com/blog/Daily-Recaps-in-Excel/

Deal-Based Watchlists

Often times, our clients are asked to value assets with few, if any, market observations. Under such scenarios, the best available data can be obtained by creating a deal-based watchlist. Insights on the equity or mezzanine tranches can often be gleaned from activity higher up the stack. More information on this watchlist creation method can be found at the following link: www.empirasign.com/deal-tracker/

Leverage Third Party Cash Flow Tools

With the click of a mouse, the Empirasign web application can export our Market Data to cash flow tools such as INTEXCalc, YieldBook, and Bloomberg. You can read more about this here:

• www.empirasign.com/blog/YieldBook-Excel-Add-In-Integration/
• www.empirasign.com/blog/Deep-Links/

High Variance Marks

Our High Variance Marks report shows all N-PORT bonds with the highest valuation standard deviations. This report highlights the bonds the market cannot price with high precision, and will help you determine what exactly is a "good enough" level of precision for a particular bond. Related Blog Post: www.empirasign.com/blog/High-Variance-Marks/

Outliers

This report highlights all statistically significantly outlier valuations for all N-PORT bonds. If your bonds show up on this report, it's probably a good idea to double-check your math and be prepared to defend a mark that is well above or below consensus.
Related Blog Post: www.empirasign.com/blog/Surfacing-Outliers/

Enterprise Solutions

In addition to the web application tools, our data is available via other channels that are more suitable to larger groups or teams. In practice, many teams license both the web application and one of the following bulk delivery methods.

Flat File Delivery

If your shop has its own data warehouse and prefers the simplicity of a flat file ETL process, we can deliver multiple cuts per day of all the Market Data we collect.

On-Demand API

Our Market Data is available via a language agnostic REST API. This means your team can use Excel VBA, Python, Java, or any other language to extract Market Data from Empirasign. Below is a link to our Developer Resources page:
www.empirasign.com/developer-resources/

Conclusion

We discussed the data sets and tools Empirasign makes available to ensure your valuations are correct and computed in a time effective fashion. If you would like to discuss further please reach out to us at [email protected] or 646-837-8848. If you prefer to cut out the middleman, and head on over to our Free Trial Registration Page.

Deal Documents Full Text Search

Many of you have your own deal documents library. Most likely this is just one big shared directory of inconsistently named and sporadically updated PDF files. By using the Empirasign Deal Documents Library, you can spend less time on this unwanted avocation and more time on your core tasks of providing value to the markets and your customers. The Empirasign Deal Documents Library consists of a variety of document types. These include Offering Memos, Prospectuses (and Supplements), Term Sheets, and Indentures (coming soon).

Today we extend the functionality of our 15,000+ document library by offering Full Text Search into these documents. If you're one to Cut the Corner on a dogleg, head right on over to www.empirasign.com/deal-docs/ to get started right away. This page can also be found under the Query Tools section of the Navbar.

A Few Motivating Examples (Words and Phrases to Deal Docs)

• Are you curious how others are financing purchases of assets yielding 0% with liabilities yielding 5%? Simply search for the term bitcoin.
  www.empirasign.com/deal-docs/?fts=bitcoin
• Which developers refuse the take a side in Pickleball Vs. Tennis Culture War? Search for developments that have both tennis and pickleball courts via the search term pickleball AND tennis.
  www.empirasign.com/deal-docs/?fts=pickleball+AND+tennis

Full documentation for search methods can be found by clicking on the ? icon above the full text search box. Additionally, these instructions will autoload any time a malformed query is entered.

Example Full Text Search Results Screen

Search results page for pickleball and tennis

• The Search Engine Results Page shows a snippet for each document (up to 100) matching your search string.
• Click on the View Document button, and you'll be taken to the exact page in the document where your search query matches.
• Click on the Search Trade Data button, and you'll be taken to a page showing related trade data (BWICs, Runs, NAIC, TRACE, and Holdings) for all the tranches in the matched deal.

Identifiers to Deal Docs

Our new Full Text Search functionality is not the only way to surface and view Deal Documents. In the left search box, you can search for Deal Documents via ticker, CUSIP, or ISIN. Additionally, you'll find links to all available Deal Documents on any secid or BWIC page, and our real-time New Issue Monitor. An example screenshot from a BWIC page is shown below. The links are circled in red.
BWIC 250430G

That's about all we have for now. Please watch this space as Full Text Search just marks the beginning of our foray into the world of financial prose (unstructured data). This is the next step after spending the last 15 years taming the world of financial numbers (loosely structured data).