[340,000 Locations Worldwide] Weathernews Takes on the Challenge of Global Data Collection

At Weathernews, we launched a global data aggregation project in 2023 with the goal of delivering highly accurate weather forecasts to people around the world. Nearly three years since its inception, we now have access to data from 340,000 locations worldwide.

How do we collect this vast amount of data, and how do we apply it to our daily forecasts?

In this article, we'll take a closer look at the behind-the-scenes efforts of Weathernews' data collection initiative and our three-year journey toward building the world's largest weather database.

Access to Weather Observation Data from 340,000 Locations Worldwide

Why Weathernews Collects Data from Across the Globe

The driving force behind this project stems from a core commitment: to be a company that doesn't simply learn about disasters through TV or the internet, but one that detects anomalies early and continuously monitors the latest conditions firsthand. This reflects our pride as a weather company.

In recent years, typhoon damage across Asia has become increasingly severe and impossible to ignore. Faced with the loss of lives and significant economic impacts, we felt a deep sense of responsibility.

To help improve safety worldwide by leveraging the forecasting technologies and disaster communication expertise we've developed in Japan, we launched this global data collection project in 2023.

A Three-Year Challenge

The Journey and Struggles of Building the Database

Our project team set a three-year plan with a simple goal: collect as much data as possible. We focused on gathering the following types of data: • Forecast models operated by international organizations • Warning and alert information issued by countries worldwide • Data such as rainfall, wind, temperature, and humidity collected by observation devices installed by public and private entities • Rainfall intensity and wind data observed by meteorological radar systems operated by public institutions

Some of this data is commercially available, while other datasets are distributed free of charge.

Warning data was relatively easy to obtain, and we've now achieved a population coverage rate of 98%.

Paid datasets were also relatively straightforward to acquire. With formal agreements in place and fees involved, strong support from providers enabled steady progress.

However, the most challenging part was working with free, publicly available data. In many cases, there was little to no explanation of the data itself—for example, whether it represented "10-minute rainfall" or "hourly accumulated rainfall."

Attempts to contact data providers were often unsuccessful. Phone calls went unanswered, and emails sometimes received no response. Without support from the responsible personnel, we ultimately had to manually verify and interpret the data ourselves—an extremely time-consuming and painstaking process.

Acquiring Areal Rain Cloud Data Through MoUs with Asian Countries

Some datasets, while publicly available, required significant time and effort to obtain. A prime example is meteorological radar data.

Although many organizations publish rain cloud images on their websites, what we truly need lies upstream of those visuals. To improve forecast accuracy, we require numerical data—such as geolocation and radar intensity—rather than processed images.

In such cases, we work directly with meteorological agencies in various countries to obtain the necessary data, enabling its integration into our forecasting models.

Last year, Weathernews strengthened its partnerships across Asia by signing a Memorandum of Understanding (MoU) ¹on mutual cooperation in the meteorological field with the Vietnam Meteorological and Hydrological Administration, as well as a collaboration agreement on meteorological cooperation with the Philippine Atmospheric, Geophysical and Astronomical Services Administration (PAGASA)².

There is still a vast amount of data worldwide that remains inaccessible. Moving forward, Weathernews will continue to strengthen partnerships with these organizations to promote data sharing, enhance our data collection efforts, and improve forecast accuracy—ultimately contributing to the safety of people around the world.

Rigorous Quality Control

Accelerating New Value Creation with Weather Data × AI

We're also investing heavily in building the infrastructure needed to accurately analyze vast amounts of weather data from around the globe and turn it into practical value.

The first critical step is data quality control. If anomalous values are detected in incoming datasets, they're automatically flagged as errors, allowing them to be immediately excluded from forecasting calculations and training data.

If incorrect data were used, it could degrade forecast accuracy and even compromise the performance of our machine learning models. More importantly, it could lead customers to make decisions based on inaccurate information. That's why rigorous quality control at the entry stage is essential.

Next, we perform primary data processing. In reality, the weather data collected by Weathernews varies widely in file formats and internal structures. Simply looking at the raw data doesn't clearly indicate what it represents or what specific numbers and symbols mean.

To address this, we standardize all data into a unified format using common terminology, ensuring that anyone can easily understand the content at any time. In this process, we place particular importance not only on human readability but also on machine readability, ensuring that AI can process the data instantly without confusion.

Creating an environment where both humans and AI can properly handle data is the key to generating new value.

To fully leverage this refined data, we've developed an in-house AI agent. Tasks that once required specialized expertise, such as data search and extraction, prototype development for new services, and workflow automation—can now be performed by non-engineers simply by interacting with the AI using natural language.

This environment has dramatically accelerated the speed of value creation within our company. Today, many of Weathernews' services incorporate AI, and behind the scenes, new ideas are continuously being brought to life through this AI agent.

Although data quality checks and preprocessing may seem like modest efforts, each step forms a crucial foundation that enables Weathernews to continuously generate new value from vast amounts of weather data.

Building the World’s Largest Observation Database

Delivering Highly Accurate Forecasts to Everyone

To accurately understand weather phenomena worldwide, Weathernews has continuously expanded its observation network. Over the three years since 2023, we've successfully collected data from 340,000 locations.

By combining this vast dataset with our proprietary forecasting technologies and cutting-edge AI, we'll continue to deliver more accurate forecasts and innovative weather content to people around the globe.

Our mission is to empower individuals and businesses worldwide to leverage weather information for safety and growth. We'll continue to expand the scope and variety of our observational data and strive to create new value for society.