If Federal Reserve Bank (FRB) data is free, why pay for it?

Actually, FRB data is, in fact, free.

Background: Test data was needed for another database project and the Federal Reserve Bank is a good source. We didn’t need everything they have. However, it turned out all FRB statistical releases are fairly consistent being distributed on top of the SDMX standard.  Since we already built an automated ETL (Extract, Transform, and Load) process to download and handle FRB data according to that standard, we figured why not do all of them.

Here are the main reasons FRB_SR is a good value:

1) The FRB website limits downloads in a fashion that is a bit cumbersome and disorganized.
2) The FRB data downloads are not in a readily usable format.
3) The FRB and SDMX folks, being (quasi) government agencies, have a tendency to be a little careless about making “unannounced changes” . . . four times in the last six months . . . which are show stoppers. Adjusting for this is a relatively easy fix for us since it is a simple code change . . . and we are use to it.
)4 ROI . . . FRB_SR is cost effective.  Over the past many years we have designed, developed, maintained, debugged and reverse engineered countless data applications and systems . . . based on our experience an organization will spend thousands PER MONTH to built and administer a similar process.

So, the fee is not for the data but for doing the ETL process, packaging it in a real database, and distribution.

 

 

Federal Reserve Bank Data Quirks and what to do about them.

Quirks and “Undocumented Features”

The Federal Reserve Bank has established a pattern of introducing quirks to the data or the XML schema.  These quirks break down their entire technical approach to their data download service.  Fortunately, the sound architecture of FRB_SR greatly aids in compensating.  However, traces of the Federal Reserve Bank’s missives still appear in the data.  For example, the Federal Reserve Bank added about two dozen series code numbers to the “Flow of Funds Accounts of the United States (Z.1)” statistical release without the code names.  Those series code numbers were manually added but the code names were added like “UNKNOWN . . . temp fix for 123456”.  When the Federal Reserve Bank gets around to updating the XML schema or publishing the series names, this issue will be corrected.

Simple and Compact without the Quirks

We designed FRB_SR in a very compact form to facilitate distribution.  Like the Federal Reserve Bank, other data providers offer very segmented and limited access to the data.  FRB_SR contains every statistical release the Federal Reserve Bank publishes.  To make distribution possible, the schema is very simple with only three tables.  Seven data views provide a basic breakout and access to the statistical releases and their sub-levels.  While the FRB_SR design has performance features, the schema is a more compact data warehouse rather than a high performance database.  The initial execution of a view  will likely yield slower performance which improves significantly thereafter.  We have found this adequate for our  development and testing purposes.  If you find  performance is an issue in your data environment, we recommend extracting the data out to fully optimized data tables.  We are currently working on “expansion” scripts to do this.

When can I get clean data without the Quirks?

We publish FRB_SR version updates monthly around the fifteenth for the prior month’s data.  Federal Reserve Bank dictates the schedule.