In part 1, we explained what data chunking is about in the context of scientific data access libraries such as netCDF-4 and HDF5, presented a 38 GB 3-dimensional dataset as a motivating example, discussed benefits of chunking, and showed with some benchmarks what a huge difference chunk shapes can make in balancing read times for data that will be accessed in multiple ways.
In this post, I'll continue looking at that example dataset to see how we can derive good chunk shapes, generalize to other datasets, look at how long it can take to rechunk a multidimensional dataset, and look at the use of Solid State Disk (SSD) for both accessing and rechunking data.
Version 4.2.6 of the netCDF Operators (NCO) has been released. NCO is an Open Source package that consists of a dozen standalone, command-line programs that take netCDF files as input, then operate (e.g., derive new data, average, print, hyperslab, manipulate metadata) and output the results to screen or files in text, binary, or netCDF formats.
The Unidata Program Center wishes a fond farewell to budget analyst and multitalented team player Tina Campbell, who is embarking on a new career with an environmental research and consulting firm.
No one enjoys a mushy tomato. But beyond aesthetics lies a serious problem for some West African nations, where significant portions of the fresh tomato harvest may be lost during transport from farm to market.
Spatial agro-ecologists from the Faculty of Geo-Information Science and Earth Observation (ITC) of the University of Twente in The Netherlands and elsewhere have used the IDV to analyze the relationship between observed weather conditions andthe microclimate inside truck trailers transporting tomatoes.
The European Geoscience Union (EGU) conference held in Vienna, Austria from 7-12 April, 2013, will feature several sessions dedicated to scientific visualization. In addition to a regular session within the ESSI program, there will be a "splinter meeting" that will concentrate on a demonstration of visualization tools. Attendance is limited, so interested individuals should sign up for the session soon.
Google Earth Outreach is offering two workshops geared toward higher education institutions, focusing on tools that are useful for undergrad and grad-level teaching, coursework, and research in GIS and remote sensing. The workshops are intended for mapping and technology specialists looking to "collect, host, analyze, visualize, and publish map data using the power of the cloud."
