Events

Now that I've got a few weeks without travel, performances*, or work conferences, I can go back to not having enough time to read all the news that interests me. Like these stories: The Economist examines how Putin might be punished for war crimes in Ukraine. Max Boot wonders why Tucker Carlson still loves his old Uncle Vlad. The IPCC says we have eight years to cut greenhouse emissions by 50% or the planet will pass the 1.5°C warming threshold no matter what else we do. Welp. Via Bruce Schneier...

On Friday, I used Arithmetic™ to predict that the 162-million-row weather data transfer from Weather Now v3 to v5 would end around 7pm last night. Let's check the logs: 2022-04-04 18:48:30.7196|INFO|Clearing v3 archival records for ZYTX 2022-04-04 18:49:27.7471|INFO|Moved 157,408,921 weather archives from v3 to v5 2022-04-04 18:49:27.7471|INFO|Finished importing; duration 4.04:14:55.0952715 Nice prediction. (It logged 157 million rows because I made a performance tweak and re-started the app after 5...

I realize posting has slipped a little in the past couple of weeks. It should resume its normal frequency tomorrow, as I actually have five consecutive weeks of a routine schedule coming up. That routine includes rehearsals on Mondays, though, so nothing new today.

The Apollo Chorus performed last night at the Big Foot Arts Festival in Walworth, Wis., so I haven't done a lot of useful things today. I did take a peek at the other weather archive I have lying around, and discovered (a) it has the same schema as the one I'm currently importing into Weather Now 5, and (b) it only goes back to August 2006. Somewhere I have older archives that I need to find... But if not, NOAA might have some.

As of 17:16 CDT, the massive Weather Now v3 to v5 import had 115,441,906 records left to transfer. At 14:28 CDT yesterday, it was at 157,409,431, giving us a rate of ( 41,967,525 / 96,480 seconds = ) 435 records per second. A little more math gives us another 265,392 seconds to go, or 3 days, 1 hour, 43 minutes left. So, OK then, what's the over-under on this thing finishing before 7pm Monday? It's just finished station KCKV (Outlaw Field, Clarksville, Tenn.), with another 2,770 stations left to...

Two stories this morning seemed oddly juxtaposed. In good news, the City of Chicago announced plans to spend $15 million on 77 km of new bike and pedestrian trails over the next couple of years: Several of the projects, including plans to convert an old railroad into a trail in Englewood, are still in the planning and design phases. Others, like Sterling Bay’s planned extension of the 606 Bloomingdale Trail into Lincoln Yards, are set to come to fruition through private partnerships.  The news release...

It takes a while to transfer 162.4 million rows of data from a local SQL database to a remote Azure Tables collection. So far, after 4 hours and 20 minutes, I've transferred just over 4 million rows. That works out to about 260 rows per second, or 932,000 per hour. So, yes, the entire transfer will take 174 hours. Good thing it can run in the background. Also, because it cycles through three distinct phases (disk-intensive data read, processor-intensive data transformation, and network-intensive data...

The data transfer from Weather Now v3 to v5 continues in the background. Before running it, I did a simple SQL query to find out how many readings each station reported between September 2009 and March 2013. The results surprised me a bit: The v3 database recorded 162.4 million readings from 4,071 stations. Fully 75 of them only have one report, and digging in I can see that a lot of those don't have any data. Another 185 have fewer than 100, and a total of 573 have fewer than 10,000. At the other end...

Sunday night I finished moving all the Weather Now v4 data to v5. The v4 archives went back to March 2013, but the UI made that difficult to discover. I've also started moving v3 data, which would bring the archives back to September 2009. I think once I get that done then moving the v2 data (back to early 2003) will be as simple as connecting the 2009 import to the 2003 database. Then, someday, I'll import data from other sources, like NCEI (formerly NCDC) and the Met*, to really flesh out the...

I won't belabor the point, or even inject my own opinion about Will Smith's Oscars meltdown Sunday night, except to say I'm amazed at how many articles, columns, and Tweets have appeared about it. I guess nothing else in the world matters right now?