Streamlining Data Transfers for Educational Testing Services Company

Raising the bar in education: A case study on implementing an online assessment and administration framework to enhance student achievement in public primary and secondary schools.

Business Problem/Scope of Work

The Educational Testing Services Company faced challenges in managing the transfer of candidate registration and test data, including responses, scores, and other related information to other vendor systems for a series of standardized assessments administered to public primary and secondary schools in the states of Nebraska and Alaska. These challenges included manual data entry, delays in the assessment process, and difficulty integrating different systems used to retrieve test metadata, candidate data, and test session data. To address these issues, Pronix developed an online assessment and administration framework to automate the data transfer process, improve efficiency, and provide easy access to all relevant data in one place. The goal of the project was to improve the overall assessment process and provide more accurate results for students in Nebraska and Alaska.

Business Solution

The Educational Testing Services (ETS) company developed a Candidate Data Transfers framework to address the challenges faced in managing the transfer of candidate registration and test data. The framework aimed to improve the overall efficiency of the assessment process by providing a comprehensive solution that streamlines the data transfer process.

The framework featured a process with multiple phases and steps, allowing each phase to be configured at the level of the test program.

It also supported multiple test programs within the same deployment, had the ability to schedule unattended automated execution, provided internal web monitoring and control UIs, used APIs exposed by other components, limited direct access to other component databases, and processed multiple candidates simultaneously, to expedite the transferring process.

Technical Solution

To implement the Candidate Data Transfers framework for Educational Testing Services, the company decided to utilize a number of Amazon Web Services (AWS) to provide a highly scalable and reliable solution.
One of the key components of the solution was the use of Amazon Elastic Compute Cloud (EC2) instances. These instances allowed ETS to deploy their Java applications to AWS using their existing application deployment tools and processes, or to integrate Java application deployment with automated deployment tools and services such as AWS CodeDeploy or AWS OpsWorks.
To store the candidate admin records files, We made use of Amazon Simple Storage Service (S3), an object storage service that offers industry-leading scalability, data availability, security, and performance. S3 was chosen as it provides the highest durability, ensuring that the candidate records would be safe and secure.
We also made use of Amazon Simple Queue Service (SQS) as a message queuing service to enable distribution of load in CDT BAT instances. This allowed for the dynamic provisioning of the optimal quantity and type of compute resources based on the volume and specific resource requirements of the batch jobs submitted.
The implementation included spring batch job and for horizontal scaling, and considering the need for future scalability, the use of partitioning (using Partitioner and PartitionHandlers) was chosen over remote chunking. A spring task executor was also used at one node to execute steps in parallel.
The output format used in the implementation is JSON format to store the project dependent property and value pairs within the database. A Groovy utility class was also used for validation of the entire XML with XSD. The reader reads all records, partition into the number of partitions specified and invoke independent processes to process and write out to the CRDS interface.
The asyncTaskExecutor of the Spring framework would be used for this now in a single tomcat instance. This technical solution allows ETS to scale and manage the data transfer process effectively and efficiently, while also providing high availability and security for the candidate data.

Technologies/Skills Used

We implemented the solution with a combination of programming languages, frameworks, development tools, databases and operating systems to develop the Candidate Data Transfers framework. The primary programming language used was Java 1.8, with Groovy being used for specific functionalities. The frameworks used were Spring (Core, Web Services, Batch), Apache Common Chain, Hibernate (4.x and 5.x), Log4j and EhCache. Development tools used were Maven for project management, Jenkins for continuous integration, SVN for version control, JIRA for project management, and Confluence for team collaboration.

The database used was Postgresql, which is a powerful and reliable relational database management system. The framework was deployed on AWS (Amazon Web Services) and made use of S3 and SQS for storing and managing data respectively. The operating systems used were Cent OS and RedHat Enterprise Linux.

Customer Success Outcomes

The implementation of the Candidate Data Transfers framework for Educational Testing Services resulted in a number of positive outcomes. The key outcomes were: ability to onboard new customers, ease of maintenance of the application using AWS monitoring services and improved performance by serving 190K concurrent users. These improvements allowed educational services company to expand their customer base, improve the ease of maintenance and provide better performance, which in turn helped customer to provide better service to their customers and students.

Latest Case Studies

Our Case Studies

Pronix is committed to protecting and respecting your privacy. Please confirm that you agree with our privacy policy by checking the box below.

* I agree with the privacy policy and consent to receive communications from Pronix.