- Added new Y-axis options - new tests administered, and new positive tests
- Resolved some recently created CSS layout problems with the header on the main page
- New data for 5/28 loaded from JHU CSSE and covidtracking.com (US testing data)
- Moved "US Counties" out to its own page and reworekd it to leverage postprocess better. Bottom line - you can now change the selected counties and even change the y-axis option without re-dispatching the entire search
- Added new data for the populations of all US counties. This allowed a new y-axis options in the "US Counties" page, for Total Confirmed Cases by % of population
- New data for 5/25 and 5/27 and loaded from JHU CSSE, covidtracking.com (US testing data) and Aura Vision (lockdown dates)
-New data for 5/24 loaded from JHU CSSE and covidtracking.com (US testing data)</li>
- New data for 5/22 and 5/23 loaded from JHU CSSE and covidtracking.com (US testing data)
- Added buttons to the main tab with which the user can launch the Job Inspector, export results as csv, or print the page.
- New data for 5/21 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/20 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/19 loaded from JHU CSSE, covidtracking.com (US testing data) and Aura Vision (lockdown dates)
- New data for 5/18 loaded from JHU CSSE and covidtracking.com (US testing data)
- Fixed a bug where if you didn't normalize the _time axis it would give you one bucket per month instead of one per day
- New data for 5/16 and 5/17 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/15 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/14 loaded from JHU CSSE and covidtracking.com (US testing data)
- reordered the y-axis options, partly to improve organization but also to make the "remaining patients sick" option not be the default any longer.
- Moving the logarithmic axis primer into the notes section since log axes are no longer the default and since they're only enabled for a couple of the y-axis options
- New data for 5/13 loaded from JHU CSSE and covidtracking.com (US testing data)
- Modified the Logarithmic axis option within the US Counties tab to disable and hide itself if statistics are selected for which it makes no sense. This matches the same fairly longstanding behavior in the main tab.
- New data for 5/12 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/10 and 5/11 loaded from JHU CSSE and covidtracking.com (US testing data)
- "US Counties" tab - Performance improvements.
- "US Counties" tab - Added new y-axis options of "deaths - per day" and "cases - per day"
- "US Counties" tab - removed the "remaining patients sick" and "patients who have recovered". Since the recovered stats are generally not very well reported at the county level, these were misleading.
- Added new y-axis options to the main tab - "tests - total tests administered" and "tests - percent testing positive"
- New data for 5/9 loaded from JHU CSSE and covidtracking.com (US testing data)
- A little cleanup in the search that feeds the CheckboxPulldown for countries/provinces/states, roughly halving the render time for those options
- New data for 5/8 loaded from JHU CSSE and covidtracking.com (US testing data)
New data for 5/7 loaded from JHU CSSE and covidtracking.com (US testing data)
- Some more cleanup and layout improvement within "Recovery by country".
- pulling in a couple fixes that just went up on covidtracking.com (US testing data)
- Some cleanup of the "Recovery by country" tab, as well as introducing a second chart for each selected country/province/state, that shows the actual Deaths per day over time, for reference
- New data for 5/6 loaded from JHU CSSE and covidtracking.com (US testing data)
- A bit of cleanup in the analysis view just makes the entire first tab actually run as a datacube+postprocess pattern. this vastly improves the speed at which the charts render there.
- Fixed a bug where the granularity of the timecharts in the 'recovery by country' tab was snapping to weeks instead of days for some countries/provinces/states with longer histories.
- New data for 5/5 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/4 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 5/3 loaded from JHU CSSE, covidtracking.com (US testing data) and Aura Vision (lockdown dates)
- Fixed a CSS bug where the text color and the background color of the navigation menus was the same and you had to mouseover them to read them at all
- Commenting out the exponential-forecast section. There are so few tracks that are still following exponential curves that even with its heavy qualifiers, it's probably misleading more folks than it's helping.
- Changing Deaths and Confirmed y-axis options to default to linear scale instead of logarithmic scale
- Fixed a CSS bug where the bottom paginator of the 'recovery by country' tab was partly hidden behind the last chart
- New data for 5/1 and 5/2 loaded from JHU CSSE and covidtracking.com (US testing data)
- Finished the work to get population numbers into the underlying search result, by country and also for US states.
- added a y-axis option to the UI to display the total number of confirmed cases as a percent of the population instead of as a raw count.
- New data for 4/30 loaded from JHU CSSE and covidtracking.com (US testing data)
- Fixed a cosmetic bug introduced yesterday where the NavBar's links in this app became a dark blue color that was quite hard to see against the grey background
- New data for 4/29 loaded from JHU CSSE and covidtracking.com (US testing data) and netlify.com (lockdown dates)
- New data for 4/28 loaded from JHU CSSE and covidtracking.com (US testing data)
- The app now checks to see whether the data it has is more than a day old. If it is, it gives the user a warning and tells them about the three ways they can update the app to get newer data
- New data for 4/27 loaded from JHU CSSE and covidtracking.com (US testing data) and netlify.com (lockdown dates)
- New data for 4/26 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today
- New data for 4/25 loaded from JHU CSSE and covidtracking.com (US testing data)
- New data for 4/24 loaded from JHU CSSE (JHU didn't post 4/24 numbers last night so it slipped until today) and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
- New data for 4/23 loaded from JHU CSSE and covidtracking.com (US testing data) and netlify.com (lockdown dates)
- New data for 4/22 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
New data for 4/21 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
- A new landing page exists.
- New data for 4/20 loaded from JHU CSSE, covidtracking.com (US testing data) and netlify.com (lockdown dates)
- Rewrote python scripts to use requests (formerly it was just a quick wget inside popen). Note that the app does not run refresh.py itself, but if the local admin were to manually run it (splunk cmd python refresh.py), then it would pull down fresh data from the three sources.
- New data for 4/19 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
- Added a new "Deaths per day, 3 day moving average" option for the y-axis
- Added some plain english statistics to the "recovery by country" tab.
- New data for 4/18 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
New data for 4/17 loaded from JHU CSSE, covidtracking.com (US testing data) and netlify.com (lockdown dates)
- Fixed a bug where the header for the main chart would get misaligned if certain combinations of options were selected
- New data for 4/15 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
New data for 4/15 loaded from JHU CSSE and covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
- Fixed a bug in how the default selected countries were being set, that regressed with the fixes last night for France/UK
- New data for 4/14 loaded from JHU CSSE, covidtracking.com (US testing data) and netlify.com (lockdown dates)
- The main tab now has an option where you can choose to not normalize the y-axis at all. If you pick this it does give you some options for where you want the time axis to start.
- There is also a new option to normalize the time axis to "days since local state/country locked down", using the lockdown data that the app has been pulling in from netlify.com
- there is a new statistic that you can pick for the Y-axis, "cases - percent growth by day"
- Fixed bugs when you would try to select all provinces for France or UK, and get just the actual provinces, and not the "main country" Province.
- Fixed a minor bug where the US would appear in a slightly wrong place in the sort order.
- The Chart is now vertically resizable
- New data for 4/13 loaded from JHU CSSE -- it seems they corrected a typo for the case count in Okaloosa County Florida today (4/13). Cases were earlier today listed as 102103 instead of 103.
New data for 4/13 loaded from JHU CSSE, covidtracking.com (US testing data) (no new lockdown data from netlify.com today)
- New data for 4/12 loaded from JHU CSSE, covidtracking.com (US testing data). (The lockdown data wasn't updated today on netlify.com )
<strong>0.9.1 (April 11th, 2020)</strong>
<li>New data for 4/10 loaded from JHU CSSE, covidtracking.com (US testing data) and netlify.com (lockdown data).</li>
- Reimplemented how the app gets and uses the JHU CSSE data. The app now ships a) a single lookup that is a merged file containing data from all of the daily report files. b) a script refresh.py that can be used to refresh the master.zip from github and then rebuild this lookup from that zip's daily report files. NOTE - The app itself does not run this script yet daily. That will likely happen in a day or two though.
- Completely rebuilt the "recovery by country" tab to instead display a chart of the "Expected Rise In Deaths" statistic for every selected country, so you can page through one chart for every country you've selected.
- Fixing some inconstencies in how Hong Kong's data was treated.
- Fixed a bug where the "reset selection to..." control only appeared if you were in the first tab.
- Note: Minimum required version of Canary is now 1.2.6
- Fixed appinspect failures that were just silly packaging mistakes.
- New data for 4/9 loaded from JHU CSSE
- Removed the temporary fix that was in place for the covidtracking.com testing data, and instead now there's simpler code that just makes the file a local lookup directly.
- Adding a new lookup that has lockdown dates by country and state. Although this is not used in the UI, this *is* used in the underlying SPL now. To make a long story short there is a field now in the main base search called "days_since_lockdown_started" for the given Country or State. As such, an intrepid if not necessarily fearless user of this app could click 'see search syntax' and then make the modifications to change the final chart command to use this as the x-axis.
- Updated the search feeding the "Current numbers" tab, to actually display all the things that have been added in the past couple weeks. This means the number of columns is about 3x what it was and it's a big horizontal scroll now to see it all. These are the breaks.
- New data for 4/8 loaded from JHU CSSE
- New data for 4/7 loaded from JHU CSSE
- Replaced the new "NetInflux" statistic that was added earlier today, with a similar but more useful one. It is explained at some length by contextual messaging if you select it in the UI. However in a nutshell it uses the region's calculated case-fatality-rate to calculate the number of new confirmed cases that day, that are expected to die. Then it simply subtracts the actual deaths from that day. In the end you have a metric that somewhat looks ahead and tells you whether deaths will be lower in a few weeks.
- Fixed a bug where if you picked one of the test counts as your y-axis, the logarithmic y-axis option wasn't getting disabled
- Added a new statistic, for the moment called NetInflux. If you select it in the UI, the app itself gives some explanatory text about what it is trying to do, and how it is calculated.
- New data for 4/6 loaded from JHU CSSE
- New data for 4/5 loaded from JHU CSSE
- New data for 4/4 loaded from JHU CSSE
- Fixed a bug where if you had 'treat provinces/states as countries' set to "no", that the reset links to show all states/provinces within US/Australia/Canada/etc did not work properly
- Bumping up the required version of the Canary app, so as to pull in some important additions from that side.</li>
- Some wording improvements to the form element options
- New data for 4/3 loaded from JHU CSSE
-- New data for 4/2 loaded from JHU CSSE
-- The experimental "forecast" panel now only appears if one of the statistics that actually does have an 'exponential' phase, is selected -- ie total deaths, total cases, and total 'sick'. Conversely if something like Case Fatality Rate is selected, it hides itself.
New data for 4/1 loaded from JHU CSSE - includes the US data that JHU hadn't put out yet when the normal daily report file was put out at 4:50pm pacific (which is fine. It's amazing this dataset is what it is, and updates every day).
- Added a new y-axis option of 'case fatality rate (then-current)'. This option switches you to a linear y-axis and disables the logarithmic y-axis. (h/t to Ari for the conversation that indicated this was a bit of a hot button statistic.)
- Added new y-axis options to allow charting of new deaths per day, as opposed to cumulative deaths. likewise new confirmed cases per day as opposed to cumulative cases per day.
- Fixed a bug - the shortcut links to reset selected countries/and states now preserve your x-axis normalization options
- in the underlying SPL there are new fields in the underlying datacube, of median_age by country (or for US States, for that state), also Population and "Urban_Percentage"
- minor improvements to organize the todos and notes and link to them from the nav
- Added a median_age field to the us_states.csv
- New data for 3/31 loaded from JHU CSSE, but this is a VERY THIN daily report and for instance it includes no new points for any US states
- A small change to condense the form elements slightly to help users with smaller monitors.
- Manually preventing France and the United Kingdom from being split apart into province/state values because they only have a couple small province/state values being broken out and since they are far smaller in population than the respective "most of the country" value, the net effect on the UI is a little confusing. and I'm going to revisit this later to find a better solution.
- added a new static lookup for countries, that has population, land area, median age, and 'urban percentage'. This isn't wired up as anything in the UI yet at all, but it will be soon.
- (New data for 3/31 loaded from JHU CSSE)
- Edited the 'default' reset link to add Belgium and the Netherlands.
- Fixed a bug in the new US-counties prototype, where the state pulldown was initially selected to "all"
- Fixed a bug where extracting the county names from jan/feb rows, inadvertently trimmed off some city-level rows. Now the group that captures the county names is optional in the regex.
- Added back the testing options to the y-axis pulldown (these pull from the covidtracking.com data, and while the machinery has been in the app for some time, I inadvertently removed the options from the y-axis several releases ago)
- Added a contextual disclaimer about the "PendingTests" statistic, that these numbers are not widely reported so the chart can be misleading.
- (New data for 3/30 loaded from JHU CSSE)
- Added some higher numbers to the "by number of days since the" selector. Unfortunately it is sometimes useful to normalize the x-axis to these much higher numbers now.
- Moved the layer of reset links into its own Pattern file.
- Added a new prototype that allows the same sort of interactivity but specifically for the counties within one particular US State. For all non-US users out there, unfortunately the JHU dataset recently started providing this County-level detail only for US States. For what it's worth when similar granularity is available for other countries I will certainly try and expose that as additional functionality, as I've done here for the US.)
- (New data for 3/29 loaded from JHU CSSE)
- Added a disclaimer to both "cases over time by country or state" and "recovery by country" to call out that the "Recovered" numbers in the dataset are not broken out by US State. Without this disclaimer it seemed potentially quite alarming. Hopefully the JHU CSSE folks are able to soon break this down by state and I can change this disclaimer or remove it later.
- (New data for 3/28 loaded from JHU CSSE)
(New data for 3/27 loaded from JHU CSSE)
- Fixed a misconfiguration of the main view, where the main Country/Province/State pulldown would reset to the default every time you reloaded the page (rather than using the selection preserved in the url)
- minor fix so the chart doesn't redispatch when you just change the y-axis type.
- Expanded the 'reset to default' link to also allow you to reset to selecting all the provinces within the few countries that have that level of granularity. This is only a convenience but it saves a lot of time selecting/deselecting provinces and states.
- Also I have added another static lookup to handle normalization of various Province_State names that have typos or inconsistencies.
- (data for 3/26 loaded from JHU CSSE)
- Fixed a minor bug where the explanatory note above the main chart always said the xaxis showed days since the Nth "confirmed case", even when that option was set to 'deaths' instead.
- (New data for the past day loaded from JHU CSSE)
quick fix for a regression in the 0.7 build just posted, where 'recovery-over-time' tab didn't load.
Splunkbase restricts release notes to only 1000 characters and the notes for this release exceed that number.
However for complete release notes for this and all releases, you can always refer to our website.
- Improved the `get_count_of_days_since_nth_case` further, so now it can take three optional args. See comments in macros.conf
- Added the ability to explode out the countries/provinces and treat them as separate countries. This allows a much more flexible analysis - notably you can compare individual US states or Australian or Canadian provinces to entire countries. The previous tab doing per-state analysis has been deleted.
- Added controls to allow the user to change from the x-axis being "days since the 100th confirmed case" from 100 to any N, and from "confirmed case" to "death".
- Fixed a bug where clicking 'see raw search syntax' on a tabular output would dump you out into a column chart
- Pulled out the primer on logarithmic y-axes into its own page under 'notes'
- Upped the required version of Canary to 1.2.3, and added a little message that appears in the top of the analysis view, if the version is too low
Fixed a problem in the "COVID-19 dataset" tab where the covidtracking.com data rows were appearing there, but with most columns blank. As this was confusing and served no purpose I filtered them out
Fixed a regression in the "COVID-19 dataset" tab where the selection of the province/state element had no effect on the results.
Added some text underneath the main tab giving users a primer on exponential curves and how to interpret charts with logarithmic y-axes
Since the JHU CSSE data suddenly has a whole other field for US County, but the field name strangely is "Admin2", I've renamed the field to "County" and made it visible in the Covid19 tab.
Fixed an unfortunately bug that only happened on 7.3, but that prevented users from being redirected into the app correctly. Instead they were stranded on a page telling them that "TopNav" and "AppNav" modules didnt exist.
Reviewed and made improvements to how we handle days with missing data at the country and province level. Previously the SPL screened out the duplicate records where the "Last Update" referred to a day before the day-based filename the record was in. Now we let those records get indexed and returned at search time but at search time we override the _time value from the filename. Normally this is a bad practice in Splunk and it's better to set it from the filename at index time with INGEST_EVAL. However here it's quick, its effective and we pull the entire set off disk anyway with each search so there's not much of a downside. Also INGEST_EVAL would force reindexing for all upgrades to this release.
Removing the warning message added in 0.4.2 and the app now simply dedups the data explicitly on search.
There is a new health checks page, whereby the app ships stanzas in checklist.conf that get run explicitly on this page. Note the checklist.conf stanzas will also run in the Splunk Monitoring Console if any splunk admins are looking there
(New data loaded from JHU CSSE dataset)
JHU recently went back and changed something in the daily data from 2 days ago, which is great. However this comes before I had completed the health check to automatically detect this and bubble up a warning to the user
So this release just has a hardcoded warning at the top to always reindex the data after updating to a new version.
Redoing the default country selection. The short version is "I added Switzerland".
The Long version - Previously it was the top 10 total confirmed cases, plus an extra 2 - Hong Kong and Singapore which are low but have extremely interesting curves. New it is the top 10, plus an extra 3 - Japan, Hong Kong and Singapore.
the app now ships a snapshot of the per-day testing data for US States from https://covidtracking.com/api/
some key numbers from the us testing data can now be chosen as y-axis values on the per-province/state chart, although of course they only work for US states at the moment.
In the "by province/state" tab I have marked the "Recovered" yaxis option as "(doesn't work with recent US data)", The reason is that since the "Recovered" numbers in the US are no longer associated with individual states, this means we can't really graph that number in the US split by state. While you might think we can create a fake state called 'unknown', we'd ahve to also determine "on what day the unknown state passed 30th confirmed cases" and that's nonsensical.
Altered the app so that even if the user fails to install the Sideview Utils app or the Canary app (or both) this app itself will explain that problem to them on the landing page and give them instructions for how to install the missing apps. (Before this version the app would just load a blank page and leave the user stranded.)
Tested on Splunk 7.3 and since everything seems fine, I am marking it compatible with 7.3 (prior to this it was marked in SB as Splunk 8.X only)
gave the navigation bar a link to the release notes file.
- (New data loaded from JHU CSSE dataset. I try to push a new build asap when there's new data on github)
unfortunately Splunkbase truncates release notes at only 1000 characters so our notes for this release do not fit in this field.
However you can read the release notes for this release as well as all others over on our website - https://sideviewapps.com/apps/covid19-reporting/release-notes/
(i need to backfill release notes but right now since it's still been languishing many days in the SB manual review queue, it's not a pressing task)
Development of this app began on March 14th and is proceeding quite rapidly. I plan on posting daily updates, as this app actually has a current snapshot of the CSSE COVID19 dataset contained inside it.
Once you install the app and restart Splunk, the app will create a new index called "covid19" and index the daily_report data (a copy of which ships within the app itself) into that index.
Check back soon for updates. Email firstname.lastname@example.org with any questions/comments.
Splunk AppInspect evaluates Splunk apps against a set of Splunk-defined criteria to assess the validity and security of an app package and components.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps and add-ons from Splunk, our partners and our community. Find an app or add-on for most any data source and user need, or simply create your own with help from our developer portal.