According to the NYT Bits Blog, a new 1.4 terabyte flash memory card has the potential to democratize access to “big data” analysis. The card, now costing only $4,000, and likely to be heavily discounted, is supposedly 25,000 times faster for a server to access than traditional disk memory. The result is that data intensive analysis can be done much more quickly, because data can be access and then processed much more rapidly.
Its time for us to start analyzing our web logs, our court files, our case management systems aggressively and in real time. This should speed triage and simplification. Maybe we need a “boot camp” in big data as part of the next TIG conference. Or maybe access to justice groups should cooperate in a training program. Think what an ATJ Commission could do with big data skills.
I like the idea of a “Big Data” Boot camp. Things to cover could include:
a. What is bid data? What is not “big data”?
b. How do you work with big data?
c. What type of analysis can you do when you have big data capacity (machine types) and certaint type of analytical tools (visualization, graphs, math, algorithms)?
d. In what type of problem (math defined) do legal problems fall in? What type of analytics would we need to do big data analysis on predictive legal analysis? Other type of big data legal analysis?
e. Who has the machines to run big data models for the legal community? What research partnerships could be created with those who have HPC (high performance super computing) capacity? Do we need HPC? Or can we use desktop/cluster models instead? Hybrid?
f. How would the legal non profit community afford this type of capacity? Who would you hire/what skills would you need to bring in house? And then how do you support it going forward?
g. Any policy/4th ammendment concerns for goverhment instituations doing big data analysis with individual information? Privacy dicussion w/experts.
This could be a very exciting meeting to lay a foundation and figure out what it would take to do this well–maybe something to follow a research conference, or combined with a research conference. Thanks for raising it Richard.