Analytics Development

Getting it right the first time

During the past five years, data analytics have been transitioning from mainly scientific and academic applications to a much wider commercial/business setting with complex integration points, such as data, technology, business processes, other analytics, and security. These previous academic/scientific “clean-room” approaches for analyzing data do not translate well to a government setting, where an agency might have several thousand analytics, working against a large number of data sets, and tightly integrated with dozens of mission-critical process/workflows. Also, academic analytical techniques typically are not concerned with addressing the data V-forces (volume, variety, velocity, veracity, variability, and visibility) that affect Big Data situations, nor with security (of varying dimensions).

Our experience has shown that data analytics is a different class of software. As software, data analytics require a different development lifecycle and corresponding methodology. There are some unique aspects of data analytics that require refinement of generic agile practices to address the unique requirements of data analytics.

At Axiologic Solutions, we have created a Data Analytics Development Methodology – ANALYZEit™ – as a way to build useful data analytics the right way. ANALYZEit extends existing systems engineering and software development practices and explicitly focuses on data analytics, specifically pitfalls to avoid, and important issues on which to focus.

ANALYZEit features

  • Multi-Disciplinary – We integrate practices from a variety of disciplines:
    • Software/Systems Engineering – for creating analytics that scale, perform, are deployable and secured
    • Computer Science – for techniques on how to organize/structure data and how to efficiently process this data algorithmically
    • Statistics and Mathematics – for techniques on how to analyze the data
    • Science Domain – for techniques specific to the domain of the data or its application
  • Multi-Domain – We use techniques associated with different data domains, such as:
    • Geospatial data
    • Numerical data
    • Text data
    • Video data
    • Audio data
    • Cyber related data
    • Graphs/networks
  • Analytic Perspectives – We use different techniques for each of the four primary classes/perspectives of analytics:
    • Descriptive Analytics – Use statistical/stochastic techniques to basically describe current (historical) business performance
    • Diagnostic Analytics – An attempt at getting to the root cause with traditional statistical techniques.
    • Predictive Analytics – Use of mathematical techniques/models to predict outputs/outcomes.
    • Prescriptive Analytics – Computationally determine a set of high-value alternative actions or decisions given a complex set of objectives, requirements, and constraints, with the goal of improving business performance.
  • Analytic Processing Styles – Techniques for the three primary analytical processing styles:
    • On-line – where a user (or a workflow) triggers some analytic
    • Batch – where raw data is accumulated (file, database) and based on some trigger (specific time of day, elapsed time, size of file, number of records, some other workflow trigger) the entire raw data set is processed at once.
    • Streaming – as individual data records are produced or become available for processing, they are individually processed by some analytics.
  • Big Data scale – These are different from regular analytics as they are address the Big Data V’s.