Probabilistic Short-Term Sky Image Forecasting Using VQ-VAE and Transformer Models on Sky Camera Data.

Overview

1. Department of Informatics, Clausthal University of Technology, 38678 Clausthal-Zellerfeld, Germany; (S.N.); (A.R.)

Abstract

Cloud cover significantly reduces the electrical power output of photovoltaic systems, making accurate short-term cloud movement predictions essential for reliable solar energy production planning. This article presents a deep learning framework that directly estimates cloud movement from ground-based all-sky camera images, rather than predicting future production from past power data. The system is based on a three-step process: First, a lightweight Convolutional Neural Network segments cloud regions and produces probabilistic masks that represent the spatial distribution of clouds in a compact and computationally efficient manner. This allows subsequent models to focus on the geometry of clouds rather than irrelevant visual features such as illumination changes. Second, a Vector Quantized Variational Autoencoder compresses these masks into discrete latent token sequences, reducing dimensionality while preserving fundamental cloud structure patterns. Third, a GPT-style autoregressive transformer learns temporal dependencies in this token space and predicts future sequences based on past observations, enabling iterative multi-step predictions, where each prediction serves as the input for subsequent time steps. Our evaluations show an average intersection-over-union ratio of 0.92 and a pixel accuracy of 0.96 for single-step (5 s ahead) predictions, while performance smoothly decreases to an intersection-over-union ratio of 0.65 and an accuracy of 0.80 in 10 min autoregressive propagation. The framework also provides prediction uncertainty estimates through token-level entropy measurement, which shows positive correlation with prediction error and serves as a confidence indicator for downstream decision-making in solar energy forecasting applications.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

The paper links to its data, not to its authors' code: see the Data section.

Tracing map

A tracing map links a paper to the code its authors published: this paper has none, so it has no map.

Data

Datasets cited

• zenodo:18657514 — at Zenodo; found in “Data Availability Statement”

Data Availability Statement

The dataset used in this study is openly available in Zenodo. https://doi.org/10.5281/zenodo.18657514 accessed on (16 February 2026).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 29 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 3 authors, 5 keywords, 1 funder, 39 references.

Cite

This paper

Seyidbayli, C., Nezakat, S., & Reinhardt, A. (2026). Probabilistic Short-Term Sky Image Forecasting Using VQ-VAE and Transformer Models on Sky Camera Data. Journal of imaging, 12(4), 165. https://doi.org/10.3390/jimaging12040165

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-74002-2 [code]

Learning missing physics from legacy simulators with alternating neural integrators.

Journal: Nature communications

In common: 2 references

[2] doi:10.3390/biomimetics11080522 [code]

Mechanics-AI: A Bio-Inspired Physics Intelligence Pipeline for Cross-Domain Engineering Prediction and Sustainable Design.

Journal: Biomimetics (Basel, Switzerland)

In common: 1 reference

[3] doi:10.3389/frai.2026.1785867 [code]

A generalized logistic-logit function and its application to multi-layer perceptron and neuron segmentation.

Journal: Frontiers in artificial intelligence

In common: 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.