We Spent Months Cleaning SEC EDGAR 13F Data So You Don't Have To

The data isn't the hard part. Cleaning it is. SEC EDGAR is public, free, and a mess. Every quarter, 13,000+ institutional investment managers file Form 13F, disclosing their U.S. equity holdings. In theory that's a beautiful dataset — every hedge fund, every pension fund, every bank, all in one place. In practice, the raw filings actively fight you: The same institution files under different names, sometimes even in the same quarter. In our own database right now: BlackRock, Inc. and BlackRock Inc. are two distinct CIK registrations for what most people would call "one" institution. Multiply that across 13,000+ filers and you get a long tail of near-duplicate names that break any naive GROUP BY institution_name. CUSIPs don't map cleanly to tickers. Foreign issuers frequently use CUSIP prefixes that don't resolve through the usual reference data — we ended up building a fallback resolution path (Yahoo Finance lookups plus manual validation rules) just to keep ticker c