Tuesday, April 25, 2017

Arc Collector 2: Creating your Own Database, Features, and Domains for Deployment and Use in ArcCollector

Background
With the completion of the previous exercise, students were now familiar with the workings of ArcCollector. With this knowledge, each student was tasked with developing a personal project, building the necessary geodatabase, feature classes, domains, and collecting the data individually using ArcCollector. When setting up the project, designing and implementing the domains and fields of features was critical. Domains are what limit what kinds of data can be recorded in attribute fields of feature classes. Without these, the possibilities for for data entry error are numerous. In addition, if the domains and fields are not assigned the proper field type, proper spatial and statistical analysis cannot be performed.
For this project, the focus of the data collection was on the bike racks located on the University of Wisconsin, Eau Claire Campus. In the last three months, construction began on the Garfield Avenue project, resulting in this area being cleared for construction. This included removing the available bike racks located in this area. With this being the case, the survey focused on the question "at what bike racks are students parking their bikes on campus and how much available space is at each location?" This survey was designed to identify the areas with the bike rack locations with the least and greatest maximum storage and compare it to the amount of the space being used.
Study Area
The study area for this survey would encompass most of the University of Wisconsin, Eau Claire campus (Figure 1). In addition, this data would be recorded at two separate times during a school day (Monday-Thursday), once at 9:30 AM- 12:00 PM and once at 6:00 PM - 8:30 PM. This would allow for the comparison for the differences between usage of bike rack at the beginning of the school day and the end of the school day. in addition, data would be gathered recording the weather and date of recording. Ideally, this survey would be collected daily over multiple weeks to get a proper survey and rule out possible outliers in addition to recording changes due to day of the week and weather. However, with the limited time available for this exercise (April 20-25), three viable days for data collection (April 20, 24, 25), and schedule constraints on the one surveyor, it was decided that data would be recorded once for each bike rack in the morning and once in the evening. The evening data points would be collected on April 24 (and April 25 for one location that was initially missed), and the morning data points would be collected on the morning of April 25. The weather and date recordings would serve as footnotes for those who were interested in looking over the data.
Figure 1: A map displaying the survey area for the ArcCollector bike rack usage survey completed on Arpil 24 and 25, 2017. In addition, the map also shows an area currently cleared for construction which has displaced a number of the bike racks, among other surface features.

Methods
First, a project geodatabase was created in ArcCatalogue. Within the Geodatabase properties, the domains were accessed. Several domains were generated to limit and monitor the future data collection:

  • Bike Number: A long integer domain for the number of bikes currently present at a given bike rack. The range was set to 0-400, showing that a bike rack could be empty up to containing 400 bikes. This range was determined as it was roughly 60% larger than the maximum possible storage for a bike rack located on campus. This would allow for data collection beyond the current limit if the University ever decided to expand the storage at a given location
  • Maximum Occupancy: A long inter domain for the maximum number of bikes a given rack was designed to hold. The range was similar set to 0-400 as the bike number domain, for the same reason. The bike number domain at a location could exceed the maximum occupancy if students stored their bikes in a way the rack was not originally designed to.
  • Time: A long integer domain for the time of data point recording for each bike rack, with a range of 0-2359. This would allow for the recording of military time in a way that it could be easily separated later for analysis.
  • Weather Conditions: A coded value text domain for the weather conditions at the given time of recording. This was set to codes for Clear (<50 cloud="" cloudy="" cover="">50% cloud cover), Rain(>90% cloud cover and raining), Snow (>90% cloud cover and snowing), and Other (Sandstorm, hail, other unlikely forms of weather).
  • Two other domains for Date and Notes were created but ultimately not implemented, as they interfered with data entry for these fields. Students were instructed to avoid utilizing a Date domain due to the complexity and difficulty in implementing such a domain.
Once the geodatabase was created, three feature classes were created in the geodatabase:
  • A point shapefile for bike racks
  • A polygon shapefile for the construction area on Garfield Avenue
  • A polygon shapefile denoting the primary areas on campus, and the most likely locations for bike racks owned and maintained by the University to be located.
These shapefiles were projected into the WGS 1984 Web Mercator Auxiliary Sphere Projected Coordinate System, as this projection worked best with ArcGIS online and ArcCollector. In the properties of the bike rack feature class, additional fields were created for the data which would be recorded:
  • Bikes: Long integer data field for the number of bikes at a bike rack. Connected to the Bike Number domain.
  • Max_Occupancy: Long integer data field for the maximum occupancy of a bike rack. This was connected to the Maximum Occupancy domain.
  • Time: Long integer domain for the time of data point collection, in military time. This was linked to the Time domain
  • Weather. Coded text field for the weather conditions at the time of data point collection. This was linked to the Weather Conditions domain.
  • Date: A text field for the date of data point collection, recorded as mm/dd/yyyy. This was not linked to a domain, but was given a maximum number of 10 characters in length.
  • Notes: An additional text filed for any notes or critical observations on a given data point or bike rack. This field was generally left blank.
Once these fields were created, the feature classes were opened into an ArcMap viewer with the same coordinate system as themselves. An editing session, along with a basemap were used to generate polygons for both the construction area and the study area around campus (Figure 1). In addition, the fields of the bike rack feature class were tested for validity and to make sure they were maintained by the domains. Once verified, the basemap was removed and the data frame was uploaded to the UWEC personal enterprise account as an editable, updatable, and remotely syncable  service. This service was additionally made to be only personally accessible.
In ArcGIS online, the feature layer was saved as a remotely editable map. This allowed for the insert and removal of data to any of the feature classes online via computer or remotely via cellular device or tablet. Once this was completed, the data could be collected. As previously described, all but one of the data points for the bike racks in the evening (6:00 PM- 8:30 PM) were recorded on April 24, and the locations and corresponding fields were collected a second time on the morning of April 25. This was done remotely via ArcCollector on a mobile device. Once the data collection was complete, it was viewed in ArcGIS online. With this, an online, interactive, and public map was generated to display the data. In addition, the online map document was opened in ArcMap, and the feature classes saved offline. This data was utilized to created a series of maps which displayed the most critical results of the study.


Figure 2: An interactive map displaying the data collected during the bike rack usage ArcCollector Survey on April 
24-25, 2017, on the University of Wisconsin, Eau Claire campus. The pairs of points are a result of collecting data for each bike rack twice, once during the morning and once in the evening. Data displayed also includes the study area and area of construction on Garfield Avenue.  
Results
Looking at the results of the survey, several conclusion can be made(Figure 3). The number of bikes on racks in the western portions of campus don't really change, regardless of the time. However, racks in the northern and eastern portions of campus appear to hold more bikes during the morning hours than the evening. This was expected, as the eastern and northern portions of campus are primarily occupied by lecture halls which are more active during the morning to afternoon. What is surprising is the seemingly lack of change in the western bike racks, which primarily surround dorm halls. Only a small number of bike racks show a decrease easily noticeable decrease between 9:30 AM and 12:00 PM, a couple of racks located near center of the study area. This would imply that the increase in bike storage in the eastern and northern portions of campus, during the day, are largely a result students who live off campus biking to school in the morning. In regards to the number of bikes stored at a given located, it seems to be tied to the size and accessibility of the building next to the rack, in addition to the visibility of the rack. The more openingly visible racks next to large buildings hold the greatest number of bikes, which cannot be easily seen by looking at a map. However, these individual numbers matter less if the total possible storage is not looked into. Without knowing the maximum available storage, it cannot be determined which racks are filling up.
Figure 3: A map depicting the number of bikes stored ate bike racks on the University of Wisconsin, Eau Claire Campus. The data was collected from April 24-25, 2017, and shows each bike rack during the morning (left) and during the evening(right).
When looking at the maximum possible storage at bike racks (Figure 3) on campus, it can be seen that many offer tremendous storage. The largest bike racks can hold anywhere from 97 to 249 bikes. These largest racks are typically located by themselves in central locations. The smaller racks generally are located on the periphery of campus, and are sometimes in small groups. The mid-sized racks are scattered throughout the remaining areas. What's important with this data is that it can be used mathematically with the number of bikes at racks to calculate the available storage that was used.
Figure 4: A map depicted the maximum possible occupancy of bike racks on the University of Wisconsin, Eau Claire campus on April 24-25, 2017.
When looking at the percentage of available storage used (Figure 5), it can be seen that most of the available space is not completely used during either the morning or evening. However, there exist several locations where the available storage was either completely used or exceeded. In these cases where the available storage was exceeded, bikes could be found coupled to nearby trees, signs, and poles. When compared to the total available storage, it was found that these filled areas also have the smallest possible maximum storage of all the bike racks. This means that these locations fill up quickly. Whats strange is that all but one of these four locations are filled throughout the day, and are located next to an almost empty rack. Only one location whose storage is used completely empties at the end of the day. Its located on the far eastern border of campus. In addition, it can be more clearly seen that areas to the north and east empty at the end of the day, while most racks in the center and western portions of campus remain relatively constant from the beginning of the day to the end. In response to this data, a recommendation could be made to increase the available storage at the filled or nearly filled locations.
Figure 5: A map depicted the percentage of available storage used on bike racks on the University of Wisconsin, Eau Claire campus. This data was recorded on April 24-25, 2017, and shows bike racks during the the morning and evening. Percentage used was calculated by dividing the number of bikes present by the total available storage. An IDW interpolation was utilized to create a continuous surface to display the data.


























Discussion
Proper database creation and domain field implementation allowed for the proper data collection and analysis completed within this project. Without it, the bike number and maximum occupancy data could not have been properly displayed and analyzed. This study, up to this point, shows that bike rack occupancy increases in the eastern and northern portions of campus early in the pay and empties near the end. The southern, central,  and western portions change little throughout the day. A small number of bike racks completely fill up, and these should be increased in size to fit the demand. However, these bike racks have the smallest maximum occupancy limit, and the largest bike racks which are shown to hold the greatest number of bikes do not fill completely. Unfortunately, weather data wasn't able to be properly explored, due to the limited time available for data collection. Ideally, this study could be repeated, with data collecting taking place over several weeks. That way, proper statistical analysis could be performed for weather data in conjunction with the bike rack usage data.

Tuesday, April 11, 2017

ArcCollector Part 1: An Introduction to Gathering Geospatial Data on a Mobile Device

Introduction
With the introduction of powerful and easily obtained smartphones, GPS units have largely been replaced when collecting location data. Additionally, smartphones have the ability to access recorded data on the go and monitor survey progress through the study area as its continually updated. In this lab exercised, one such smartphone based survey system would be introduced and utilized, ArcCollector. ArcCollector is an Esri based mobile GIS application allowing for the recording of multiple attributes of survey locations based on predetermined domains by many surveyors at once. Domains are essentially rules that limited what type of data, quantitative or qualitative, can be entered into an attribute field, and what range or descriptive data may be entered.
Study Area
The study area was the University of Wisconsin, Eau Claire campus, particularly the areas surrounding the primary area. As the area was far too large for a student to survey completely in the allotted time, the study area was divided into seven zones, labeled numerically, and each student was assigned to help survey a particular zone (Figure 1).
Figure 1: A map of the study zones of the March 29, 2017, micro-climate survey of the University of Wisconsin, Eau Claire campus. Each zone was assigned two or three students from the class to survey, and positional and micro-climate data would be recorded for each zone using the smartphone application ArcCollector.
There were two or three students assigned for each area, and each student would need to survey roughly twenty points to effectively survey the entire study area. Zone 4 was the zone personally assigned. Using ArcCollector, students gathered positional and micro-climate data from within these zones.
Methods
First, each student was tasked with downloading the smartphone application ArcCollector to their personal smartphone, available on both the Google Play and Apple app stores for free. While this was downloading, the data attributes were overviewed, so each student would know what data they were collected. This was primarily micro-climate data, with supplementary time and group data.
The attributes were:

  • Group. Listed in the attribute table as GRP. This would show what group collected the data point
  • Temperature. Listed in the attribute table as TP. This would be collected with a personal Kestrel pocket weather meter in degrees Fahrenheit.
  • Dew point. Listed in the attribute table as DP. This would also be collected with the personal Kestrel pocket weather meter in degrees Fahrenheit.
  • Wind chill. Listed in the attribute table as WC. This would be collected with the personal Kestrel pocket weather meter in degrees Fahrenheit as well.
  • Wind speed. Listed in the attribute table as WS. This would be collected with the personal Kestrel pocket weather meter in miles per hour.
  • Wind direction. Listed in the attribute table as WD. This would be collected by using a compass, either physical or electronic, to measure the direction the wind was coming from in degrees, with North being the designated 0.
  • Time. Listed in the attribute table as Time_. This would describe the time each point was collected during the day, and would be listed in military time with the colon removed.
  • Notes. A general notes tab for use later. We were instructed to keep this area blank when  collecting data
In order to normally collect data of this kind in ArcCollector, a Geodatabase with the appropriate shapefiles for both the group zones and the survey points must be created, with the appropriate attribute fields and corresponding domains, and uploaded to ArcGIS Online for group use. For the purpose of this exercise, the geodatabase and its shapefiles had already been constructed, uploaded to, and been made available on ArcGIS online for the class. Each student needed to only been given access by the instructor.
Once each student had access to the Online ArcGIS database, they were sent out to gather data points for locations within the zones. This would create point data for the micro-climate point shapefile feature class within the geodatabase. Each student had between 4:10 PM and 5:00 PM on March 29 to gather points data for the feature class. By pushing the "add point" tab within the application, it would create a new datapoint within the shapefile. It would then ask to fill in the attributes listed above. The TP, DP, and WC fields all had domains requiring the input to be a whole number in the normal range of temperatures for this time of year within Eau Claire County. The WS was limited by a domain requiring the input to either be a whole number greater than or equal to 0. The WD field was limited by a domain requiring the input to be a whole number between 0 and 359. However, do to an error created when generating the domains, the time and notes field were not properly limited. The Time field was supposed to have a domain that required the input to be a whole number between 0 and 2359, military time without the colon. However, there was no such domain present. As a result, the field ended up populated by correctly formatted entries (ex: 1636), incorrectly formatted entries (ex: 16:36, 4:36), and null values. In addition, the domain set for the Notes field would demand a form of entry if the field was accidentally accessed during the survey. As a result, this field was occasionally populated by an assortment of values (anything a student could consider a note on the location) and the proper null values. Each student completed roughly 20 survey point in their allotted area before returned to the lab.
Upon the completion of the survey, the data was viewed in ArcGIS online. Using the built in functionalities, it was possible to view the point density as a heat map, for reference. Once it was certain that all data recorded was present, the database was downloaded from ArcGIS online and turned into an offline file geodatabase. The data from the survey was brought into an ArcMap viewer so the data points could be properly viewed (Figure 2).
Figure 2: A map showing the micro-climate survey points collected during the ArcCollector Survey on March 29, 2017, between 4:10 PM and 5:00 PM on the University of Wisconsin, Eau Claire, campus. Each student surveyed roughly twenty points for micro-climate data which including group number (based on group zone), temperature (in degrees Fahrenheit), dew point (in degrees Fahrenheit), wind chill (in degrees Fahrenheit), wind speed (in miles per hour), wind direction, and the time of point collection (in military time).
Once it was ascertained that the data could be properly viewed, it was utilized to create a variety of surface and point maps displaying the collected data.
Results
Using the point data, a surface map could be created of the temperature with the IDW interpolation method and the TP (temperature) field (Figure 3). The data collected shows that the temperature in the northern half of the area is generally below 55 °F, except for scattered points close to the river, while the southwestern and southeastern portions of the map are dominated by areas where the temperature is greater that 55 °FThese areas of highest temperature are generally located directly west of buildings in exposed areas. In addition, the points to the south-east are located at the base of a steep embankment, with elevation increasing as the location moves from the north east to the south-west.This means they received the most exposed sunlight during this time of day, increasing the temperature of these areas. The one notable exception to this trend is the area in the far south, which possesses few tall buildings and is comprised largely of sports fields. This abnormality may be explained by combining a combination of the other micro-climate attributes. However, it does not show strong correlation with either high wind speed or low dew point, meaning that explanation does not likely lie with either of those two factors.
Figure 3: A temperature surface map created of the University of Wisconsin, Eau Claire campus using the data collected in the March, 2017 ArcCollector micro-climate survey. The temperature data was collected in degrees Fahrenheit between 4:10 PM and 5:00 PM. The continuous surface map was generated using the IDW interpolation method.
A wind chill map was created by similarly utilizing the point data and the IDW interpolation method, using the WC field instead of the TP field (Figure 4). When looking at this data, it is clear it exhibits the same general trends as the temperature map. Areas of higher temperature and lower temperature for wind chill line up very well with the areas of higher and lower overall temperature. The major difference is that wind chill has a lower minimum wind chill than overall temperature, 45 °F as compared to 49 °F.
Figure 4: A wind chill surface map created of the University of Wisconsin, Eau Claire, campus using the data collected in the March, 2017, ArcCollector micro-climate survey. The wind chill data was collected in degrees Fahrenheit between 4:10 PM and 5:00 PM. The continuous surface map was generated using the IDW interpolation method.

































A dew point surface map was also created utilizing the IDW interpolation method and the survey points. The DP field was the field used in this instance (Figure 5). This data shows some very different trends from temperature and wind chill. The southwestern portions of the survey area, along with the far northern locations and an area in the east-central portion of the study area have the lowest dew points, generally less than 36 °F. In contrast, the portions with the highest dew point, generally greater than 48 °F, are located on either the central to southeast and to locations to the north -central portions of the study area. These areas likely have the highest dew point because they have the greatest ambient moisture in the air. They are either at the bottom of a steep embankment (central to southeast) where water naturally collects, or either ring the river to the north or were taken from a bridge crossing the river. In contrast, the areas of the lowest dew point either sit way to the north of the river, southwest and on top of the steep embankment, or in the flat, open area between the embankment and the river. These are the drier locations with the lowest ambient moisture in the air.
Figure 5: A dew point surface map created of the University of Wisconsin, Eau Claire campus using the data collected in the March, 2017 ArcCollector micro-climate survey. The dew point data was collected in degrees Fahrenheit between 4:10 PM and 5:00 PM. The continuous surface map was generated using the IDW interpolation method.
































Utilizing the points dataset in conjunction with the wind speed and wind direction fields, a map could be generating showing both the magnitude and direction of the wind quantitatively and qualitatively as angled, graduated symbols. From looking at the data, it can generally be seen that the wind was blowing from the southeast to the northwest. In locations where the wind was not blowing in this direction, the southeastern wind was usually blocked by either buildings or or forested areas. In addition, the areas of greatest magnitude of wind speed are either on the river, to the far west, or to the far north. These areas are the least population by obstructions which would either block or divert the wind.
Figure 6: A wind speed and direction graduated symbols map created  of the University of Wisconsin, Eau Claire campus using the data collected in the March, 2017 ArcCollector micro-climate survey. The wind speed data was collected in miles per hour, and the wind direction was collected in degrees from its point of original and later angled to show the direction it was blowing towards. Wind speed is show as graduated arrow symbols labeled in miles per hour.
While it would have been appreciated if a map could have been generated for the Time field, it is currently not possible, due to the state of the data on this field. Because of errors in both domain and field entry by both designer and surveyors, the data within the fields is not properly formatted so that it can be quantitatively or qualitatively expressed. It is imperative that domains be properly constructed and formatted to eliminate data entry error by surveyors.
Conclusion
ArcCollector is a valuable and easily accessible tool for completing groups surveys across an area too large for a single individual to properly survey. With it, surveyors can collect and record multiple types of data from various locations without the need for complex GPS equipment. Only a smartphone application is required. In this survey, dozens of datapoints were taken of location across the University of Wisconsin, Eau Claire campus by students in under and hour and contained data describing multiple forms of the local micro-climate. Additionally, it allows for the monitoring of surveyors and the survey progress by accessing the geodatabase from ArcGIS online, either using a computer of smartphone. With ArcGIS online, the data can also be effectively shared and distributed to all analysts within an organization for breakdown and study. However, it is critical that the domains for each field be properly formatted. If it is not, data entry errors can occur when surveyors generate the data, as in the case of the time and note fields of the collected data. However, if this error can be avoided, ArcCollector can serve as a valuable tool for a field data analyst.
Sources
Hupy, J. (2017). Arc Collector: An Introduction to gathering geospatial data on a mobile device, such as a tablet or smartphone. Eau Claire, WI.