- Dependable AI analysis requires clear sources, accredited guidelines, and traceable inputs.
- Lacking proof and unfinished work should stay seen.
- Corrections should turn out to be procedures to enhance future analysis.
We constructed our AI-enabled analysis course of for a reasonably bizarre purpose: consistency. Operating it taught us that consistency was solely the start.
The tougher job was preserving all of the small distinctions that make a chunk of analysis reliable sufficient to examine.
Learn one firm in March and one other in September and it’s surprisingly straightforward to finish up with two impressions fairly than two comparisons. The questions drift. The emphasis modifications. What one firm taught you impacts what you discover within the subsequent.
So, we formalized the analysis course of. Proof from filings would stay separate from calculations. Calculations would observe outlined guidelines. Funding judgement would stay identifiable as judgement. Each firm would cross by the identical questions in the identical order.
That was the design. Then we began working it.
The expertise continues to be modest: one quarterly cycle throughout roughly twenty corporations. However what shocked me was that consistency was not the arduous half. Separating proof, calculation and judgement was vital, however not enough.
The method additionally needed to protect issues we had barely thought of: which supply ruled, whether or not a press release was provisional, whether or not an motion had truly occurred, who owned a chunk of metadata, and whether or not unfinished work nonetheless appeared unfinished.
A lot of the failures have been mundane. That made them extra helpful.
