Modernizing eDiscovery with Automated Processing and ElasticSearch
Modernizing Poultry Farms with Internet Connected Feed Level Monitoring
Little Bird Systems approached Lofty Labs to create a customer facing application to allow users to monitor data generated by their innovative audio-based sensors for feed bins in the poultry industry. The team had deployed sensors on multiple farms and were storing data in a database, but had no user interface for analytics, reporting, or even internal oversight. The team also anticipated large scaling challenges with their architecture in the short term future as they prepared a large go-to-market strategy for their IoT (Internet of Things) enabled sensors.
LOFTY'S APPROACH
Little Bird Systems team during a discovery workshop at Lofty Labs Lofty Labs worked with the team at Little Bird Systems to re-architect the pipeline of their sensor data using a standardized API (Application Programming Interface) over the internet, allowing their devices to passively push data into the cloud. The API became the standard interface for retrieving data as well, and Lofty developed an interactive reporting interface to surface insights to Little Bird System’s customers. The API and analytics application support an ACL model that allows for flexible governance of data access where Little Bird's employees and customers can access only the data they are granted, without compromising the confidential data of other customers. LBS On ScreenLittle Bird's IoT infrastructure is now poised for massive scaling and sensors are being rolled out to multiple commercial farms. The API powered dashboard is in use by LBS employees, farm employees, and integration partners to efficiently monitor and replenish feed levels. Native mobile applications are now in development, taking advantage of the API at the architecture's core to provide consistent experience across devices. These applications can be developed at a substantially reduced cost as they leverage a pre-existing cloud architecture and data access. Little Bird Systems is now confidently approaching some of the largest poultry integrators in the world with the knowledge that their automated systems can scale to meet demands of any size
Faced with a finite timeline and over 6 million pages of email to sort through to build their case, the opposing counsel had left a small intellectual property law firm in a tough spot. Looking through the data file by file with the firm's team of clerks and analysts was going to take months, if not years.
What Henry Law Firm needed was the ability to quickly filter the mass of email text data containing very specific content, between very specific custodians. Further complicating matters, the dataset was so large in volume that traditional eDiscovery platforms were prohibitively expensive with fees climbing into the mid 5-figures each month.
Knowing that ElasticSearch was the right tool to facet and search large quantities of text data, Lofty Labs built software specifically around the case and email dataset using ElasticSearch, Python, and Django. The solution started with an data cleansing process that codified hundreds of thousands of text files into consistent metadata and indexed it into a managed ElasticSearch cluster. Iterating on designs with the client, Lofty Labs consultants built workflows and tooling that allowed the client’s team to apply their query and tagging process to the dataset. Once the data was cleansed and indexed, Lofty Labs consultants provided statistical and text analysis on the dataset including de-duplication and relevance distribution.
Lofty Labs consultants were able to quickly dismiss over 50% of the dataset as duplicated content using checksum hashing and a “bag of word” analysis. This duplication analysis was complex and yielded faceted results—some content was undupiclated, while other emails were duplicated a dozen times or more—allowing analysts to deeply investigate any intentional obfuscation of information.
In just 90 days a team of two analysts were able to filter the number of items relevant to their case to just under 50,000 documents, a scant 8.3% of the original document set. Ultimately, it was only 80 documents that settled the case. Lofty Labs’ fees to develop the solution were slightly less than one month of hosting fees for comparable eDiscovery tools hosting the same volume of data. This allowed Henry Law Firm to keep the data in a searchable format for many months during the case in the event that new documents were produced, resulting in excess of 600% return on investment. After such a large success, Henry Law Firm re-engaged Lofty Labs to build similar software for document production for additional cases.
