How well can we predict solar radiation with one year’s worth of training data?

Image by author

Solar-powered energy generation is on the rise in the United States. According to EIA, the state of Texas is planning to add 10 gigawatts (GW) of utility-scale solar capacity by the end of 2022. This means that there is also an increasing need to power management, both short-term and long-term. In this case, can we leverage solar-related data to make any kind of forecast short-term and long-term help us monitor/manage/distribute electricity generation?

Solar Irradiation is a type of property which is used to measure solar radiation. There are different kinds of measurement: Total Solar Irradiance, Spectral Solar Irradiance, Global Horizontal…


Impacts of having high class imbalance

Photo by Chris J. Davis on Unsplash

It is of no doubt that text data is probably the biggest source of data available to us for any Data Science and/or Machine Learning related task. Consequently, many sophisticated and high performing algorithms have been invented to analyze text data and predict it’s sentiments. But application of more advanced algorithm doesn’t necessarily mean that our prediction is of high accuracy. We still need to go back to the basics and understand the nature of data, it’s challenges for any further processing.

I recently conducted a case study in Natural Language Processing (perhaps my first NLP project ever) to analyze…


A PyTorch Implementation of U-Net: Part I

Image by Jimmy Conover from unsplash

Seismic image gives a structural snapshot of the Earth’s subsurface at a time. A sound wave is sent to the subsurface from a ‘source’, which travels through the Earth’s layer at different velocity and gets reflected, refracted or diffracted along its way. In seismic imaging, we record the waves that are reflected back from different geological layers and stack them to create a 2D or 3D image. Because different geological layers have different physical properties, at the boundary between layers, the waves are reflected due to density contrast. There are different kinds of waves, but in imaging we are mostly…


Photo credit: Image by Author (Glacier National Park)

We spend a lot of time in data visualization in any Data Science related project. Often, these includes scatter plots, histograms, boxplot etc. But ever wondered if we were to visualize these data in a geographical context and perhaps that might help to draw some spatial relationships. If we are lucky to have latitude and longitude information, we can create such plots relatively quickly using existing libraries such as geopandas, plotly etc.

In this article, I would like to discuss about an amazingly simple and useful tool called scatter_mapboxthat utilizes latitude and longitude information to plot point set data…

Suman Gautam

Data Science/Machine Learning Enthusiast, I am also a Geoscientist/Musician/Photographer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store