Your Privacy

This site uses cookies to enhance your browsing experience and deliver personalized content. By continuing to use this site, you consent to our use of cookies.
COOKIE POLICY

Skip to main content
State Government

Unlocking Data Insights for a State Education Agency

Unlocking Data Insights for a State Education Agency

Data is an integral part of a state education agency’s ability to enable academic success and evaluate how various programs support that initiative. This reporting environment requires an intricate network of software and processes to centralize, organize, and protect the information. However, historical data can often live in a cumbersome patchwork of sources, requiring manual entry and reporting. Further, the data becomes siloed without a centralized system, complicating data storage and retrieval. So, when a state education department received a federal grant to improve its data-reporting capabilities to unlock key insights, they turned to UDig for the solutions they needed. After carefully analyzing the data they held, we helped them develop a data repository that streamlines how they operate — and modernizes how they work. 

How We Went from Ideas to Impact

The Idea

Segmented Data Environment Hindering Legislative Reporting Requirements

Our client had 13 years of data they were managing across 3,000+ data sources. Despite having this robust amount of data, they had difficulty responding to legislative reporting needs, causing delays and inaccuracies. Their legacy data was siloed, creating a cumbersome process for storing and retrieving data. Further, the data sources were all varied in quality — raw, aggregated, clean — making it nearly impossible to know which data version employees were processing.

This existing data environment meant they: 

  • Lacked a cohesive repository for storing and managing data 
  • Relied on laborious manual reporting processes across various teams and departments 
  • Were unable to quickly respond to governmental reporting demands 
  • Were unable to conduct effective data profiling  
  • Lacked transparency into their data 

The Process

Modernize and Streamline Data Reporting Capabilities

Our initial priority was helping our client meet state-legislative reporting requirements. The agency needed a data environment that enabled them to seamlessly provide on-demand and predictive analytics. Also, at any point, their reporting requirements could change, meaning their solution had to enable their teams to swiftly respond to any new criteria. To support this dynamic capability, the state education agency needed a data mart that could become the best source for any descriptive reporting on academic success. To do so, we worked in two phases: 1. Discovery and 2. Implementation.

Phase 1: Discovery 
To meet our client’s goals of upgrading and centralizing their data environment, we first had to do a data deep dive. We worked with their team to profile their data and organize it, so we understood what they had, what was valuable, and what they no longer needed. Further, we needed to identify how the data must work for them, so we could define the best build approach to meet their needs. 

Phase 2: Implementation
Once we conducted this discovery phase, we began building their data platform. We had to ensure our solutions met these goals: 

  • Create two data structures to consolidate their thousands of legacy data sources. They needed a data lake to organize historical and ongoing data sources, as well as future operational reports. They also required a relational model that contained non Ed-Fi data and Ed-Fi–based data store and allowed reporting across these two systems.
  • Design a data-ingestion process to load each of the legacy data sources into the new data structures. This process had to be seamless while enabling them to meet future reporting demands.  

As we worked with our client, we became a steadfast partner to our stakeholders and helped represent the business interests of how and why to best implement each solution.  

The Impact

A Data-Enabled Agency Empowering the Public with Actionable Information

Today, our client is a data-enabled agency that can swiftly and accurately report education-centric data at any moment. Their cloud-based data repository centralizes and stores all data, providing a cost-effective solution for predictive-reporting capabilities around academic success. With more data visibility, they can turn insights into actionable strategies. Further, thorough data profiling capabilities enable their team to better understand their data sources and how to best organize the information. They also have purpose-driven data warehouses focused on specific subjects, which they can further build out, as needed. This environment has optimized their robust analytics and data science capabilities and set them up for future demands. By building this foundation, all stakeholders — from the parents to legislative bodies — have more visibility into academic success, driving more informed decisions and actions.

  • How We Did It
    Advanced Analytics & VisualizationAssessment & RoadmapData EngineeringData ProfilingData QualitySolution Architecture
  • Tech Stack
    Azure Data LakeAzure Machine LearningAzure SQLAzure Synapse AnalyticsPythonSQL

Digging In

  • State Government

    Exploring the Future of AI at the Georgia Emerging Technology Summit: Data & AI 2024

    The Georgia Emerging Technology Summit: Data & AI 2024 was a landmark event for public sector leaders, showcasing the transformative potential of AI and data technologies. This summit brought together key figures and experts to discuss, learn, and network to enhance public service delivery through innovative technology. Keynote Highlights The summit featured insightful keynote sessions […]

  • State Government

    State IT Leadership & Partners Converge in NOLA

    Takeaways from the NASCIO 2024 Annual Conference This premier event brings together state CIOs, industry experts, and vendors to discuss the latest trends, challenges, and innovations in the business of IT for state government. This event was the largest ever as this community continues to grow and innovate for its citizens. Key Takeaways Generative AI & […]

  • People

    Digging in with David Allen, State Government Lead

    Digging In is a regular series of blog posts profiling UDig employees. We hope this series helps you get to know our team and understand why we dig what we do! Today, we are sitting down with David Allen, State Government Lead. UDig: Tell us a little bit about your background and your role at […]

  • State Government

    Exploring the Future of State Technology: Takeaways from the NASTD 2024 Annual Conference

    Last week, I attended the National Association of State Technology Directors (NASTD) 2024 Annual Conference in Minneapolis, MN. This premier event brings together state technology directors, industry experts, and vendors to discuss the latest trends, challenges, and innovations in state technology. The conference fosters the exchange of ideas and best practices essential for addressing shared […]

  • State Government

    UDig Welcomes David Allen as State Government Lead for Georgia

    UDig, a technology consulting company, is excited to announce the latest addition to our team, David Allen, who will take on the role of State Government Lead in Georgia. With two decades of technology experience in both the public and private sectors, David brings a wealth of experience and a deep understanding of the market […]

  • State Government

    NASCIO Midyear Conference | 4 Key Takeaways

    I attended the National Association of State CIO (NASCIO)’s Midyear Conference for the first time in National Harbor, Maryland. It was my first time at a NASCIO event, and what struck me most was how collaborative the NASCIO community is and how passionate these leaders are about serving their state’s constituents and employees. Each […]

  • State Government

    Improving the Process and Accuracy of State Permits and Inspections

    A custom, user-centered application that supports existing business processes, allows for function-wide automation, and converts data from a legacy system while meeting state compliance requirements.

  • State Government

    AI in State Government | Observations from Tech Summit

    The Georgia Emerging Technology Summit: AI 2023 offered valuable insights into the current state and future of artificial intelligence in state government. While many states are still grappling with the policy and ethical considerations surrounding AI, the potential for transformative applications across various departments is undeniable. Key Takeaways | AI in State Government Policy & […]

  • State Government

    Predicting Student Performance Through Machine Learning

    With the level of data collection in modern education, there is a growing interest in using this data to identify students who are potentially falling behind and pinpoint areas in which they can improve. With the extensive data collected by educational institutions, we can leverage machine learning to help educators by flagging if a student […]