Quick mapping in R: Getting elevation data for a local area. All code here http://rpubs.com/dgolicher/quick_map_edit Duncan Golicher 12/24/2018 Introduction There are now many ways of obtaining maps of the world in R in vector format. These can be useful if you want to analyse some statistics at a country, or administrative level. library(tmap) library(raster) library(dismo)…
Bayesian credible intervals for diversity indices Duncan Golicher 12/24/2018 Go to http://rpubs.com/dgolicher/bayesian_diversity for full code. Background The ideas behind the code below occurred to me way back in 2004. At the time I was researching Mexican forest biodiversity. My PhD had focussed on individual based models of forest dynamics. Although I needed to find parameters…
Analysing Likert scale satisfaction scores Duncan Golicher 2017-8-18 Full code on Rpubs http://rpubs.com/dgolicher/Limert_scale_analysis Introduction Although there are many possible ways of scoring a response to a question, the Likert scale has recently become a default option. In particular, the national student satisfaction survey (NSS) is based on questions socred on the Likert scale. NSS scores…
Cricket innings and survival analysis Duncan Golicher 2017-8-14 At http://rpubs.com/dgolicher/surviving_an_innings Introduction Yesterday I packaged up some cricket data that I had downloaded from ESPN cricinfo and reformatted. I thought it might save some time if anyone wanted the data in a convenient format. I then made a little shiny app to quickly look at batsmen’s…
Some challenges of scraping cricket data from online sources Duncan Golicher 2017-8-13 https://rpubs.com/dgolicher/crickdata_package The way that data is shared online is a constant source of unnecessary frustrations. Last weekend I was curious about the characteristics of batmen’s innings while watching the test match between England and South Africa. See my posts on RPubs. I decided…
The UK referendum on leaving the EU I’ve not been using WordPress since 2013, but I found that there is still traffic on the site, so will update it. This post was from last year and posted to https://rpubs.com/dgolicher/referendum_analysis The full post with correctly formatted R code is best viewed on Rpubs Introduction Much…
INEGI, the Mexican institute for national statistics and geography, place large quantities of data in the public domain. Geographical data is now provided through an efficient map server. However most socio economic data is only available directly in the form of Excel spreadsheets from the descarga masiva site. http://www3.inegi.org.mx/sistemas/descarga/ These data vary in quality but…
One of the fastest ways of deploying the results of PostGIS queries on the web takes advantage of Geoserver SQL views. The process is very straight forward once you have installed and configured a Geoserver. The Geoserver layer from a PostGIS data store is just defined as an SQL query with placeholders for the variables…
The recent publication of Hansen and colleagues’ global deforestation map in a user friendly Google maps earth explorer format (including street view for Mexico) has received a great deal of publicity. So I placed the Landsat classification for Southern Mexico that our group published in Google maps using more or less the same format. Click…
In my previous post I noted that the fact that the global deforesation map produced by Hansen et al has been placed on an interactive portal equipped with streetview and high resolution background imagery allows anyone to validate the results. One way the web portal could be enhanced would be to provide tools that allow…
I have received a number of comments from colleagues working in Mexico and Central America regarding the accuracy, or lack thereof, of Hansen et al’s recently published deforestation map. Many of the comments regard the timing of deforestation. Pixels shown as recently deforested were often regarded by those with ground based knowledge as having been…
In my last post I looked at some general patterns of deforestation in Central America as shown by the recent publication by Mathew Hansen and collaborators. http://www.sciencemag.org/content/342/6160/850 The data are available for interactive browsing here. http://earthenginepartners.appspot.com/science-2013-global-forest The layers have been made public, and the authors anticipate widespread use of the information. However it will not…
This week saw the publication of what may turn out to be one of the most influential products derived from processing Landsat imagery. Mathew Hansen of Maryland University and colleagues, including researchers working for Google, have pieced together a fascinating map of global deforestation over the last decade using the 28.5m pixel resolution imagery. http://www.sciencemag.org/content/342/6160/850…
Many PostGIS raster queries now run much faster than previously due to optimisation of the underlying code. However speed is still greatly affected by decisions made regarding tile size and data storage. This is an update to a post published around a year ago. Sensitivity tests are easy to run from R using the system.time…
Although it is usually preferable to stick with stable releases of PostGIS for production databases, there are good reasons to use recent development versions for raster processing. Compiling from source on Ubuntu is quick and painless using default settings, providing all the libraries are available. The loaders are compiled and just need installing. PLR can…
Back in May I participated in the Geostat summer school in Quebec by Webinar http://geostat-course.org/ This year’s course concentrated mainly on the use of R for spatial analysis, so I gave a brief tutorial showing how PostGIS can be run through R. The material is available here http://rpubs.com/dgolicher/6373 All the code can be run through…
Although it is simple to move data from PostGIS in and out of R using ODBC there are some moments when it would be nice to use an R function within the database. This is easy enough in Ubuntu. First install the postgresql-plr package and then run the query through your data base to add…
One of the motivations behind the development of PostGIS raster is the need to effectively handle time series of large raster layers. This has always represented something of a challenge for desktop GIS, particularly once the layers become too large to fit into memory. Modis NDVI data is a good example. The data can be…
One of the tasks that is commonly used in most spatial analysis at some point involves obtaining zonal statistics from a vector on raster overlay. This has always been rather inelegant using GRASS or Arc as it involves several steps. A PostGIS raster query simplifies the work flow. However in some cases my queries were…
The Worldclim data set is a fantastic resource for species distribution modelling. Not only does it include quality controlled monthly temperatures (minimum and maximum) and precipitation data but it also provides 16 derived bioclimatic layers that are designed to be universally applicable and interpretable. These layers are quite simple functions of the original data and…
Modis data are “big data” in every respect. Both the temporal and spatial extent of the information obtained from Modis makes data processing rather challenging. A quick overview of the Modis QuickFire data can be obtained by running the following code in R. A complete archive of all pixels recorded as having possible fire activity…
I have been using PostGIS (on and off) for several years now, so I am fairly comfortable with the basics of spatial SQL. However I only began looking at PostGIS raster a couple of weeks ago. My current knowledge of PostGIS raster is summarised in the figure below. I am hardly in the position to…
Los que asitieron en el curso de bases de datos especiales en Ecosur vieron unos ejemplos de analisis con capas espaciales que estan basicamente libre y en el dominio publico (con la sola restricción de registrarte como usuario con el Conabio y el GBIF. HAY QUE HACERLO ANTES DE USAR EL VINCULO A LOS DATOS…
Species distribution models now tend to use Worldclim layers by default, especially in tropical areas where historical climate datat is either sparse or unreliable. However in Mexico the CNA does provide a fairly good archive of data that can be used to derive the worldclim layers. How do the two compare? Looking just at precipitation…
El comisión nacional de agua (CNA) de Mexico tradionalmente mantuvo los datos climatalogicos en un herramienta informatica llamado “CLicom”. Todavia se encuentran datos en este formato anticuado. La exportación de Clicom produce tablas de esta forma. ¡Este no es facil de entrar en PostGIS! Pero el formato es consistente. Siempre cuando se trata de datos…
We want the data that we place in our PostGIS database to be easily extracted by students and technical staff with limited knowledge of spatial SQL. Some queries, particularly those involving raster data can be very daunting. We could try to use a mapserver or build queries for each user. However we would like to…
Acabo de dar un curso introductorio al uso de bases de datos espaciales. La materia no es muy acabada y consiste en anotaciones puesto encima de “screenshots. Espero que muestra la utilidad de PostGIS para la investigación. Los tutoriales aqui son mas relevantes a los que tienen acceso a la base de datos institucional que…
The query shown in the last post vectorised the raster in order to produce a very precise overlay and measure the are exactly. Raster points falling on the edge of the buffer would be split and only the part of the pixel falling inside is measured. If pixels are small in relation to the polygons…
Another very common operation using both raster and vector data involves extracting statistics from the raster that falls within a buffer around points. The method shown yesterday can be used to obtain mean values from numerical values. Extracting the percentage of the buffer that consists of each category of a categorical raster requires a slightly…
We are looking at PostGIS raster in the context of what is mainly desktop GIS analysis. The intention is to speed up routine operations on large data sets. However we are hoping that elegant data handling will allow us to build some useful data extractor functions that can be shared with other users. So running…
Duncan Golicher's weblog 