Federal and state agencies recently received a C- from U.S. Senate investigators on the Homeland Security Committee, failing to implement basic safeguards against cyberattacks and putting millions of Americans’ personal information at risk. Agencies face tremendous challenges modernizing their data infrastructure, the most important of which is data migration.

The data, security, and cost challenges with data include issues surrounding awareness of what data exists, its purpose, who needs access to it, and how frequently. Security challenges include issues relating to cybersecurity, data privacy, and other threats. Lastly, issues with cost include the initial cost of data migration and subsequent cloud storage fees.

The Deal With Data

Siloed data infrastructure is one impediment to smooth cloud migrations. Decades of growth and data sprawl create siloes across government agencies as well as within an agency’s own organizational structure. This is not an indictment of government but a recognition of the realities of organizational growth within multimillion-dollar organizations. Growth can create siloes if there is not persistent and consistent leadership guidance driving the collaborative interoperability conversation. When organizations elect to transition data to the cloud, they must be aware if siloed data and duplicate data is present within their internal data sets. This is imperative because, if siloed and duplicate data are not identified prior to cloud migration and are ultimately stored on the cloud, the agency could be paying to store redundant data. The siloed infrastructure also limits visibility into the true value of stored data, as IT and cybersecurity staff generally operate under each agency’s own policies while also adhering to federal regulations, state statutes, and rules.

State agencies store enormous sums of data in order to effectively operate and serve their citizens. One resounding issue across multiple agencies is either fragmented data sharing, limited data sharing, or no data sharing among sister agencies that rely on the same data sets for different purposes. This causes the same or similar data to be stored across multiple agencies, therefore duplicating data and increasing the overall storage costs for the state.

Data Security

One of the most critical concerns during cloud migration is to ensure that sensitive data is safe and secure throughout the migration process. Data migration can be a high-risk process with numerous vulnerabilities to include incomplete data migration, corrupt or missing files, accidents, phishing, malware, or ransomware.

Agencies face cybersecurity threats every day, yet few outside the government know how significant this threat is and the tremendous burden those agency teams are managing. Many agencies store so much personal identifying information (PII) that it’s easy to build out entire profiles of potential victims with background history, birthdates, addresses, and more. States are racing to implement stronger data storage compliance rules, and agencies are fighting as hard as they can using the already stretched resources they have, but many can fall behind implementing new workflows, internal processes, and access credentials. This observation is not a tacit request for more resources, which traditionally means more money for more people and tech, but rather an inflection point to advocate to approach the problem differently.

Additionally, many agencies aren’t migration experts and may struggle to keep talent that could otherwise double their salary in the private sector and be mobile in this new world of remote work. They also may not know what the latest, best practices are to safely move mass amounts of data. Phased migration is a slower yet far safer practice to avoid and fix errors and familiarize agency officials with the cloud’s security, starting with low-priority and low-risk data. Before the migration happens, backups should be securely stored for everything should a disaster recovery event occur.

Cloud Storage Expense

Many state agencies struggle with data modernization projects because policy-makers tend to follow an “all or nothing” theory that all their data must live in the cloud. When the decision is made to transition to the cloud, agencies begin the process of retroactively preparing all data for migration, migrating the processed data, and storing all the data in the cloud, only to find the costs can be more than originally anticipated. This unfortunate problem can leave agencies with significant and growing cloud storage bills. This approach also creates legislative budgetary challenges because cloud storage requirements increase as agencies keep pace with operational data growth and the related storage demands.

One of the No. 1 issues is the sticker shock of cloud storage after migrating data. All they see is the data going into a cloud storage solution and the invoice for that storage coming out, seemingly plain, simple, and expensive.

This issue is exacerbated by the fact there are very few agency leaders outside the IT/CIO shop who are aware of storage optimization strategies to cut costs and reduce risk.

Cloud storage can be tiered out according to accessibility needs. Data which needs to be accessed daily typically costs more, whereas data that’s rarely accessed, such as data that is stored for regulatory compliance purposes, is less expensive. While tiering out data by accessibility can significantly cut costs, managing data across multiple clouds can become a problem — especially if it’s replicated without a cloud strategy.

Steps for Achieving Progress

To successfully complete cloud migrations, government managers can look to enterprise organizations that treat cybersecurity as a mission critical function that’s needed throughout every part of the agency.

Some agencies are outpacing their goals. For example, Customs and Border Protection (CBP) is ahead of schedule for cloud migration. As of mid-August 2021, nearly half of the agency’s applications are in the cloud, which outpaces expectations for only a third of applications to reside in the cloud at this point in the year.

One best practice for agencies to smooth the migration and cyberthreat is improved management of identity and access control. Large agencies may be managing a patchwork of staff, citizens, and various contractors who all use data that comes from a wide variety of sources. As agencies scramble to meet cloud mandates, more challenges are exposed. Remedies include a single structure for access rights management, building "zero trust," and following best practices as established by cloud providers.

Migrating to hybrid clouds also requires a different skill set among the IT team. For groups accustomed to on-premises infrastructure, the cloud’s virtualization and new security requirements pose new challenges. Agency leadership must advance their understanding of these new environments and push hiring experienced talent with security and application monitoring expertise. As applications shift to the cloud, agencies also need development teams to produce technologies that provide a quality customer experience (CX) to drive adoption.

Federal and state agencies modernizing and transitioning to cloud storage is a significant feat that increases safe access and data value for citizens. However, it’s essential for officials to begin with understanding exactly what data they have, prioritize storage by accessibility, minimizing cyberthreats, and ensuring data sprawl is managed. While this is a different way to view the challenge because it comes with an intelligent analysis before storage approach, it is achievable. We have to first attack this problem differently or the data mountain will continue to grow exponentially as we try to chase it with more data storage funding requests.