Urban Observatory Portal goes live.

The Urban Observatory programme is deploying sensors and exploring the use of data to help inform the public about city services, and to help researchers better understand how cities work.

The data for this comes from a number of different streams and is held in a schemaless data structure. A python based middleware layer has been developed to ease the interactions with this structure. Facilitating the storing of data, the management of data and also creating different views onto the data.

It is through this middleware layer that a portal has been developed. By using django,  the Python Web framework tool. That allows data to be extracted that works with a number of JavaScript libraries e.g openlayers , highcharts.

The portal itself show the location of the sensors, provides graphs of the sensor data. As well as some extra functionality like heat mapping.

UO portal 1 UO Portal 2 UO portal 3

With the deployment of a wave of new sensors being planned in the next few months. You should see a few more dots appearing on this map shortly.

The portal is available  http://ceg-sense.ncl.ac.uk/lturf/public/

Neil – @neil_py_harris

Urban Observatory Sandpit – Sun, Sea and Seven Superb Ideas.

Wednesday 16th – Thursday 17th saw me attending my first sandpit event. Not really knowing what one was even when I was told I would be attending a sandpit event by the beach in Cullercoats I got very excited. Unfortunately I wasn’t given a bucket and spade on arrival but actually found the event probably more enjoyable and interesting than merely making sand castles on the beach for 2 days.

Dove marie Lab

The venue for the sandpit

The event was actually part of the Urban Observatory (UO)  with the aim of the 2 days to come up with ideas of pilot projects that could form part of the overall UO. The focus for the first morning was to brainstorm ideas of what data we should be recording and how we should be recording it. With a wide array of sectors being represented in the room these ideas varied greatly from transport flows, microbiology of computer keyboard, building movements and people’s motivations. In the afternoon we were asked to predict what the world will be like in 20 years time. With most group taking a pessimistic approach the outlook proposed was rather depressing. Basically climate change will only get worse, economic disparity will increase, society will divide , technology will take over people’s lives and who knows what league Norwich City will be playing in. The conclusion made was if this is to happen the Urban Observatory has to be the “good-guy” and not just record interesting information but to make recordings and carry out analysis that will help the general public and feed this data back to the general public in a manner which will be beneficial to masses.

Day 2 begun with cross-sectoral teams of three sent off to come up with an idea that represented the overlap in the 3 represented sectors of research. This is where my blog post has to get a bit cagey as it seemed every idea that came from this breakout session was either a clever or novel piece of research. The main overall theme of the ideas was public participation, either in collection of the data or in how the data was disseminated. These ideas were then honed down with more people contributing their suggestions on how an idea could be improved. As a result of this process 7 projects began to take shape. These ideas were presented to the group with more feedback coming from across the room. Some more polishing was done taking on board the feedback before again each of the 7 projects were presented. Each of which I thought had the potential for some great research and development to take place. Having not been to an event like this before I didn’t know whether this was a good haul or not. But I was told it is very rare for there to be so many good ideas which exhibit so much cross-sectoral research. Either way I don’t envy those in charge of whittling these ideas down.

I’d like to thank Rich, Phil and the other members of the UO team that put on the event. And a massive thanks to Tim and Liz from KnowInnovations for brilliantly facilitating the event.

Neil – @neil_py_harris

Making evidence useful via visualisation, 20th June 2013, NESTA, 1 Plough Place, London

Research Assistant in GeoInformatics, David Alderson, attended the “Making evidence useful via visualisation” event organised by the Reuters Institute for Study of Journalism, Oxford held at NESTA on 20th June, in London. The event organised brought together members of the academic research community in the fields of computer vision, visualisation and computing science, alongside a wealth of representation from media outlets, designers, artists and government agencies and departments. The event focussed on the need for effective visualisation of data and the challenges faced in delivery of tools and techniques to help a wide audience wade through the growing quantity of collected data. Keynote speaker Dr Luciano Floridi (Director of Research, Oxford Internet Institute, University of Oxford), and Alan Smith OBE (Principal Methodologist, Office of National Statistics) amongst others, highlighted the need for transparency and provenance of the visualisation process to allow users to understand the processes and transformations data may have undertaken to arrive at a particular visual representation. The provision of access to the underlying data, as well as the derived output or tool itself, can help to provide that lineage for a user, and potentially allow them to “recreate” the process undertaken to create the visualisation, or at least have access to the same data to create visualisations of their own. A presentation by Aleks Collingwood, Programme Manager and Statistics Specialist at the Joseph Rowntree Foundation (JRF) gave an overview of the new JRF Data portal, and echoed the sentiments of allowing users the capability to create their own versions and interpretations through links to underlying data. Finally a discussion session, chaired by Geoff Mulgan, Chief Executive of NESTA, encouraged the audience to participate in highlighting some of the key challenges to effective data visualisation and where they would like to see change in how these are delivered. A summary of the key points is listed below:

  • Improved graphical literacy of the general audiences for data visualisation, and also producers;
  • Increased levels of interactivity and personalisation of the data visualisation process, in much the same way as Google Maps has allowed everyone to become a cartographer;
  • More approaches and tools enabling user-generated content, in this instance, user-generated visualisations;
  • MORE FUN!

However, the proliferation of data visualisation techniques, tools and technologies does raise the question of at what point have we replaced “too much data” with “too many visualisations”? Will there come a point where a decision-maker will require sophisticated tools and techniques to search through a myriad of visualisations of data, rather than the data itself?

Geospatial Engineering in the Caribbean

This week myself and Neil Harris from the Geospatial Engineering group are participating in the second Technical workshop of the Cariwig project. Participants from the Climate Research Unit at UEA, the University of the West Indies Climate Studies Group, the Caribbean Community Climate Change Centre in Belize and representatives from the Instituto de Meteorología, Cuba have been discussing how best to use the web based climate data, analysis and visualisations tools developed by Neil and myself to create impactful case studies to show decision makers the potential impact of climate change across a variety of Caribbean countries and sectors.

A follow up meeting later this year will provide training in the web based tools to a large number of stakeholders in the region.

GISRUK 2014 – Céilidhs, kilts and k-means cluster analysis

From the 16th April to 18th April I attended GISRUK in a surprisingly sunny Glasgow. The started with a welcome from Jane Drummond followed by an interesting key-note talk from Mike Worboys , A Theoretician’s eye view of GIS Research. He talked about how GISRUK has developed there has been  a dramatic fall in the proportion of papers that covered the theoretical side of GIS, with the focus now being on Application.

Talks from the first day that I particularly enjoyed focused on Spacebook, a system of delivering directions via audio as users encountered various way points on a route. William Mackaness talked about his research in comparing directions given using street names and direction given using landmarks.

Phil Bartie, who was a researcher on William Mackness’s paper delved deeper into the issue of Landmarks. Using images he looked at what people identified as landmarks and then analysed them semantically and spatially to distinguish related and unrelated features.

The following day saw me presenting a talk on some of the sensor infrastructure work we’ve done at Newcastle using HStore as a key-value based approach to storing heterogeneous data. Robin Lovelace’s talk round the merits of twitter data in research. Sparking some interesting debate about how research using twitter data uses data from the public so should benefit the public.

Thursday evening then featured the conference dinner followed by the Céilidh, an event that I was secretly dreading, dancer I am not. So was pleasantly surprised at how much fun the event was, as someone pointed out to me; it’s dancing but dancing with an algorithm.

Friday morning then featured my favorite talk of the conference with  Paul Brindley talking about his Phd work on extracting the additional address information from addresse listed on the internet to map out neighborhoods. A simple idea but with fascinating results.

The conference was then wrapped up with a keynote from Professor Alias Abdul Rahman who gave an interesting overview on the state of 3D GIS work.

The event was a fun filled thought provoking and thoroughly interesting. Thanks must  go to Jane Drummond for seamlessly organizing the event, even managing the weather. I would also like to thank William Mackness who brilliantly chaired my session.

Bring on Leeds 2015. Wonder what there version of Céilidh will be?

Neil – @neil_py_harris

Strewth, these guys are SMART!

The inaugural International Symposium for Next Generation Infrastructure was held between 1st and 4th October 2013, at the SMART Infrastructure Research Facility at the University of Wollongong, Australia, and one lucky Geospatial Engineering researcher from Newcastle was able to attend. David Alderson gave a 20 minute presentation entitled A National-Scale Infrastructure Database and Modelling Environment for the UK following a successful submission of a conference paper to the conference committee, under the same title. The work contained within the paper and the presentation represented an amalgamation of work conducted by David and other researchers from the Geospatial Engineering group at Newcastle and other research institutes and universities involved in the Infrastructure Transitions Research Consortium (UK ITRC) programme. The focus of the paper and presentation was to give readers and delegates alike a glimpse of some of the work undertaken in the process of constructing a database of infrastructure-related data relevant to the UK. This included not only an overview of some of the datasets that may be found within the database, but also a preview of some of the visualisation tools that are being developed on top of the data. An overview of these visualisation tools can be found within other posts in this blog site here.

A copy of the slides can also be found here and here. Unfortunately the presentation has had to be split into two parts, so please download from both links to get the full presentation.

Other representatives from the School of Civil Engineering and Geosciences at Newcastle University, UK could also be found delivering presentations at the event including:

Miss Sarah Dunn, PhD Student – Modelling infrastructure systems for resilience and sustainability (part one, part two)

Mr Shaun Brown, PhD Student – Resilience of resource movements to disruptive events

Mr Matthew Holmes, STREAM PhD Student – How do we ensure the assessment of infrastructure resilience is proportionate to the risk?

Further to this fantastic opportunity, a further round of meetings looking to develop collaborations between researchers at SMART, including former Newcastle-based PhD student and post-doctoral researcher Dr Tomas Holderness, and the Geospatial Engineering group at Newcastle, is being held at the SMART infrastructure facility between October 8th and 11th 2013. These meetings will look to focus on potential collaborative opportunities regarding network interdependencies between infrastructure networks, and also web-based data dashboards for visualisation and dissemination purposes.

Watch this space for more information…

FOSS4G13 Geohack.

From the 17th-18th of September I attended the Geohack event at FOSS4G conference in Nottingham.  The event was sponsored by the MetOffice in partnerships with many other of organizations within Environmental Science to Services Partnership. A number of challenges were presented for hackers to work on, these are listed here http://2013.foss4g.org/geohack/challenges/ . As part of the LTURF project I have found myself working with a number of APIs trying to extract any information for around Newcastle. This lead me to picking the outdoor event challenge as it was making use of several APIs  http://2013.foss4g.org/geohack/challenges/events/ .

FOSS4G - Challengers presenting - Met Of by aburt, on Flickr

Creative Commons Attribution-Noncommercial 2.0 Generic License  by  aburt– The challenges being presented

 

This aim of this challenge was to estimate the effect that weather has on outdoor events in the UK.  The first task was to retrieve information about any outdoor events happening in a given location. This was achieved by using the skiddle.com, an events listing website, and their API.  Which allows you to search for events planned using a point and a buffer.  Once we had the events the next goal was to gain an estimate of the number of people attending each event.  This task was assigned to me, and for this I used both the Facebook and the twitter APIs. With Facebook returning the numbers of attending and maybe responses to the event, and twitter returning a popularity score.   The final step was to use Met Office DataPoint API to get the weather forecast of the event.  These were then all integrated into a very simple web portal. This is live here , we had little over 24 hours to complete this task so the portal is very much substance over style.

FOSS4G - Hacking teams working away - We by aburt, on Flickr

Creative Commons Attribution-Noncommercial 2.0 Generic License  by  aburt – Us Hard at work

 

At 1pm on the second day everybody had to stop developing and then present they work for it to be judged. It was very interesting to see what the other groups had achieved , with some of the achievements being very impressive given the amount of development time. Phonegap, which is a free and open source framework that allows you to create mobile apps using standardized web APIs across many platforms, was used by a number of groups. And is clearly a very powerful tool with some groups having a completed app developed in just 24 hours. Unfortunately my group didn’t finish in the top 3 which I’m certain is only because the complexity of our app was hidden, in short we lost because it was too good.

The event was brilliantly run with plenty of food, drink and electricity to go round and I would like to thank the members of the Met office and the foss4g volunteers for organizing the event.  And Also Steven Citron-Pousty who setup the OpenShift server for our app.

What an Asteroid Belter!

As part of the British Science Festival 2013 being hosted at Newcastle University, researchers, staff, lecturers and students were invited to contribute to a comic being developed and designed by local comic enthusiasts. The comic was officially targeting children between the ages of 8-13, with a view to promoting STEM subjects and science in general. The proposed content of the comic had, you guessed it, a distinctly scientific theme to it, with organisers interested in hearing from those staff members who could fit their work (or at least make it look like their work fits) in to one of the following themes:

  • Robots
  • Explosions and danger
  • Dinosaurs
  • Space, time and travel
  • Very big and very small things
  • Things we eat, and things that eat us
  • Codes, ciphers and hidden messages
  • Heroes and villains
  • Matters of life and death

Some of the work related to networks, connectivity and infrastructure that is currently being conducted within the context of the Infrastructure Transitions Research Consortium (ITRC – www.itrc.org.uk) and also within the School of Civil Engineering and Geosciences, seemed like something that kids could be interested in, and the idea of presenting this type of work within a comic context offered an alternative and novel mechanism for communicating our research. Once an expression of interest in contributing to the comic had been lodged, it was simply a case of meeting an artist and writer assigned by the organising committee, and beginning to think about what a comic to represent networks, connectivity and infrastructure should look like! In fact, largely due to the expertise and imagination of the artist I had been assigned (Terry Wiley), the ideas began to come together, and rather than stick to the traditional panel-based approach we decided on more of a spider diagram affair to communicate how different bits of infrastructure are connected together. However I still wish we had managed to get our idea of slightly transparent paper with networks hidden until the page was held up to the light, to fly to communicate the idea of hidden or non-obvious connections! Oh well, maybe next time. For all those interested, the comic, entitled “Asteroid Belter”,  in hard copy format was distributed at the British Science Festival 2013 (7th-12th September), but more information about it can be found here.

Everything's Connected

Everything's Connected

 

Students “flood” to measure the BlueLine

The hosting of the British Science Festival 2013 by Newcastle University gave staff and researchers of the School of Civil Engineering and Geosciences a chance to “show off”  some of their research to the general public, whilst also gave the opportunity to generate interest in science, technology, engineering and maths (STEM) subjects via hands-on activities delivered to school children from the North East. One such activity, named “BlueLine”, organised and delivered by Dr Geoff Parkin (Senior Lecturer in Hydrology), Professor Hayley Fowler (Professor of Climate Change Impacts), Dr Vedrana Kutija (Lecturer in Computational Hydraulics), Dr Claire Walsh (Researcher in Water Resources), Mr Vassilis Glenis (Researcher in Water Resources), Mr Philip James (Senior Lecturer in GIS), and Mr David Alderson (Researcher in GeoInformatics), focussed on explaining the complex nature of flooding and extreme flooding to separate groups of secondary school children from local schools in the North East. The schools represented are listed below:

–          Shotton Hall Academy, Peterlee

–          Berwick Academy, Berwick

–          Ponteland High School, Ponteland

–          Thomas Hepburn School, Felling

The sessions began with Dr Parkin delivering a brief summary of some of the causes of flooding and extreme flooding, citing many references that relate to flood events experienced in recent years within the UK, including those that occurred in Cumbria in 2005, Tyne and Wear in 2008 and 2012, and many others. However that was just the standard “in-class” part of the activity, and although the seminar generated interest from the students in such topics as the water cycle, climate change, global warming, and the impacts of flooding, the “really” interesting part came when the children were let loose outside on the University campus, to take part in a crowd-sourcing exercise.

Crowd-sourcing in it’s most simple terms involves members of the public contributing data, information, comments, stories, pictures etc, which are then subsequently used within some form of analysis. Within the School of Civil Engineering and Geosciences, there has been a concerted effort since the flooding in a local market town, Morpeth, in 2008, by water and geomatics researchers and lecturers alike to collect as much crowd-sourced based information about flooding and flood events that occur within the region. The contributions from the general public about the locations and timings of floods, alongside photography giving a reasonable indication of flood depths, levels and wrack lines, can be used to validate computer simulations of flood models, developed within the School. Some of the results of these crowd-sourcing endeavours can be found here.

So the purpose of crowd-sourcing in the context of flooding was explained to the students in attendance at the Blueline event, and how it can be used to help better understand how a flood propagates through different environments. Since the floods in 2008 and 2012, Philip James has developed an Android-compatible “app” that helps to facilitate this process, by allowing a user to use the location technologies within an Android device (tablet, smart phone etc) to locate themselves, and then subsequently use the camera to take a picture. The great thing about the app is that this data can then be sent to a server and published immediately onto a website, giving a near-real time record and subsequent archive of a flood event. As stated, this type of data is invaluable to understanding how a flood arises and then subsides. For more information about the app and the technologies employed to deliver this solution, have a look here.

So the school children were escorted outside where a “pretend” flood had been erected around Newcastle University campus (effectively just a line of blue tape indicating a flood level based on a particular amount of rainfall), and asked to use the app and a tape measure to record information about the depth of this “flood”. This seemed to go down a storm (sorry) as it offered them a chance to do all the things that children seemingly would rather be doing i.e. being outside and playing with technology! Each pairing were asked to try to record at least one reading at each point where we stopped around campus. Further to this a series of QR codes had been erected along the blue line that the students were able to scan, subsequently directing them to a site of historical photography of the flooding that occurred on campus in 2012, as well as some “constructed” imagery of the imaginary flood that they were measuring. For more information on this site and the photos, please follow this link.

BlueLine

BlueLine

BlueLine

BlueLine

The “BlueLine” around campus Imaginary flood on campus
   
Real time Flood Map Flood App Framework
 
Campus Flood Map

 

British Science Festival 2013

British Science Festival 2013

Seminar by Prof. Pascal Perez on urban simulation and planning

TransMob: A micro-simulation model for integrated transport and urban planning.

Prof Pascal Perez, SMART Infrastructure Facility, University of Wollongong, NSW

4-5pm, 24th May, Room 2.32: Cassie Building, Newcastle University

To book your place please register online

 

How will current socio-demographic evolution affect future transport patterns and traffic conditions?

How will urban development and transport policy influence the quality of life of various segments of the local community?

The ‘Shaping the Sydney of Tomorrow’ Project (StSoT) was commissioned by Transport for NSW (Australia) to better understand the interactions between transport and land-use dynamics as experienced by individuals and households over extensive periods of time (15-20 years). Stepping away from traditional optimisation, our model focuses on anticipating short and long-term emergent consequences and feedbacks resulting from interactions between people and their urban environment, through the creation of ‘what-if’ scenarios (risk assessment approach).  The innovative design and development of TransMob aims to challenge three traditional but highly limiting modelling assumptions:

•Long-term steady-state equilibrium of the system: in fact, transport services and urban development co-evolve along with socio-demographic changes in highly dynamic ways and out of equilibrium.
•Feed-forward effect of urban development on transport networks: in fact, evidence suggests that there is a strong feedback effect of transport solutions onto land-use changes.
•Homogeneous and utility-based social responses to transport and land-use planning: in fact, there is more to decisions on transport modes or residential locations than pure micro-economic reasoning; most unintended consequences stem from unexpected heterogeneous individual considerations.

TransMob is made of six modelling components: (1) synthetic population, (2) perceived liveability, (3) travel diaries, (4) traffic micro-simulator, (5) transport mode choice and (6) residential location choice. The model is applied to the inner south-east area of Sydney metropolitan area and simulates the evolution of around 110,000 individuals and 50,000 households over 20 years, according to various transport and land use scenarios.