Skip to Main Content

DAsH

Research Guide for DAsH (or digital humanities) resources and tools

Tutorials Introduction

These tutorials assume that you have access to ArcGIS either in our computer lab or installed on your computer so there won't be any instructions on how to install. They are written for someone with basic knowledge of ArcGIS's interface, that wants a walkthrough how this tool can be used to help them ask and answer their spatial research questions. 

For tutorials starting at a beginner level I recommend those on ESRI's website or if you have a New York Public Library card, they have several great tutorials for newcomers to ArcGIS on their Lynda.com portal.

The data I'll be using in these tutorials comes from felony drug arrest statistics by county that I got from the New York State Division of Criminal Justice Services, spatial data from the New York State GIS Portal and data from the Law Enforcement Support office at the Defense Logistics Agency concerning the 1033 program. In brief, the 1033 program lets local law enforcement agencies request decommissioned military equipment ranging from office furniture, to night-vision, to weapons to Mine-Resistant Ambush Proof Vehicles (MRAPs). 

While the data I'm using is real from a project that I did, it isn't going to be updated so please go back to the original sources mentioned if you'd like to explore it further. I picked these data sets because they were handy and they are large and complex enough to show how to work with different kinds of data but not too complicated to get in the way of explaining the process. The maps made in these examples are not necessarily scholarly rigorous but hopefully the process of making them will get you familiar enough with ArcGIS Pro so that when you map your own data, you'll have more time for scholarly rigor!

Learning Goals

Sometimes you'll be dealing with more granular data. Instead of a whole city, you want to map a particular address within that city, instead of a whole national park, the peak of a mountain within that park. This is point data. This can be in a shapefile of its own, in which case you'd just drag it on the canvas the same as any other shapefile. It can arrive in table form as well, as a list of addresses or as latitude and longitude coordinates, either as secondary data, or primary data you've recorded yourself with a GPS tracker.

In this exercise, you'll learn how to take a csv file with latitude and longitude coordinates, and use the Display X,Y function to add them to the map. You'll see how to use Google Maps to find the latitude and longitude for an address to revise any coordinates you may have gotten wrong and where those coordinates will be visible when your web-browser is showing information for that location.

The csv file below, geocoded-police-stations.csv contains the coordinate data that you'll be using for this exercise. 

Data

Getting Started

  • Download the csv file and place it in a folder that you can access easily. 
  • Open ArcGIS Pro, at the Welcome window, click on Map under Blank Templates to start a new project with a blank map under it
  • The Create a New Project window will pop up asking what your project should be named and the location it will be saved in. Name it NYS_PoliceStations and make sure that you have the box next to Create a new folder for this project checked. Take note of the Location it is being saved to because this is where you will find the folder containing all your project files that you will need to zip at the end of this tutorial.
  • Once your project loads up in ArcGIS, navigate to Catalog, by default, ArcGIS opens it in the pane on the right side of your screen. If you don't have it, you can add it from the View menu at the top toolbar. 
  • In Catalog, right-click on Folders and choose 'Add Folder Connection'. Maneuver to the folder that you have put these items in and click OK. It will now appear below the folder that ArcGIS created for your project, NYS_PoliceStations
  • This next part is very important for if you plan on taking your project file and re-opening it on different computers. Do not grab the csv file from the folder you saved it and drag it directly onto the map. ArcGIS Pro is very picky about file structure and if you do not keep a consistent file structure, then any layers, files, shapefiles, selections, or any other modifications that rely on being able to access those files, layers, shapefiles, tables, etc, then your project file will be defunct and all those layers will need to be added again, and any joins or other modifications will need to be redone. First, place the file within your NYS_PoliceStations folder that you'll be using.
  • Locate geocoded-police-stations.csv within the folder that you saved it to in the Catalog pane. Right-click on it and choose Copy. 
  • Close that folder, and then right-click on the NYS_PoliceStations folder and choose Paste. When geo-coded-police-stations.csv appears within that folder, then drag and drop it onto the canvas.
  • Your map won't change since it is a table and not a shapefile, but it will appear in your Contents window under Standalone Tables
  • Right-click on geocoded-police-stations.csv and select Open. Scroll through it to see the data. There are 3 columns. LEA (short for Law Enforcement Agency), Latitude and Longitude. Not a lot of information, but enough to put the items you are listing onto the map. ArcGIS can be asked to take these Latitude and Longitude numbers and turn them into points on your map.  

Displaying a Table's XY Data

  • Right-click on geocoded-police-station.csv in the Contents window and choose Display XY Data,
  • In the Display XY Data window that appears, go with the suggestions of Longitude as the X Field and Latitude as the Y Field. I'll change the end of the Output Feature Class to be NYS_PoliceStations since that will become this layer's name on the map, and I'll want to be able to identify it later. I'll leave the data about where to save it to as the same though.
  • Go with the default that it suggests for the Coordinate System in this case. However, if you are working on adding XY Data to a basemap that has its own coordinate system, that is the option that you would use to make sure that this data was working with the same coordinate system.

  • Click OK and it will display all the data on your map as a symbol for each of the different rows in your table. However, in the case of this file, it looks like a couple of the coordinates must be incorrect, since this is only supposed to be police stations in New York State, and there are data points in Pennsylvania, New Jersey and Connecticut. Sometimes you won't know that you have the wrong coordinates until you try to map the data.
  • Go over to Map and make sure that Explore is highlighted, then click on the datapoint out in western Pennsylvania. A Pop-up window will appear and let you know that this is the New Castle Police Department. 
  • I'll do the same, and write down the names of each of the police departments that were misplaced somewhere outside of New York state for me to look up manually, which I can do using Google Maps. 

Finding Latitude and Longitude using Google Maps

  • When you have written down the information for each of these, look them up either just by searching for the police station on maps.google.com or looking for that department's website, writing down the address and putting that into Google Maps. 
  • When you successfully find the correct place in Google Maps, the latitude and longitude will be part of the URL, an example of which is highlighted here. 

A marker on a map in a web browser, above it the part of the URL directly after search term /@ is highlighted that has coordinate numbers in it

The first number is the latitude and the second is longitude.

  • Go to Windows Explorer and navigate to geocoded-police-stations.csv within its file in NYS_PoliceStations.  Open the file in Excel and update the incorrectly located police stations with the new latitude and longitudes you find with Google Maps (there should be 4 you'll need to change). For the purposes of this exercise, if you can't find a police department in the town stated, pick one in the nearest town.  Save it as geocoded-police-stations_rev and as a CSV file within the NYS_PoliceStations folder.
  • Return to ArcGIS  Pro, and remove or uncheck visualization for the layer that you created with the first csv file. 
  • Right-click on the NYS_PoliceStations folder and choose Refresh. geocoded-police-stations_rev.csv will appear. Drag and drop it onto the map. 
  • Repeat the Display XY Data procedure with  geocoded-police-stations_rev.csv , but name the Output Feature Class NYS_PoliceStations_2 then run it. All the stations should now appear within NY state's boundaries

It'll just give you a default symbol for these events, but you can doubleclick on the symbol in the Contents pane and it will open the Format Point Symbol menu for you.  You can use the Gallery tab to pick the shape you want and the Properties tab to change the color and size. I'll make the symbol a blue star.

Saving Your Map

  • Click on the Project icon at the top of your menu bar and choose Save. If you are on your own laptop and don't plan on sending yourself a copy of your file to open elsewhere, you're all set. However, if you want to open this project file in another computer, or if you want to send your professor your project because it's finished or because you have a question, this is only part of what you need to do to ensure that you'll be able to access the project later. The other steps will need to occur in Windows Explorer, so exit ArcGIS Pro. 
  • Navigate in Windows Explorer to the path that your project was saved to at the beginning of this lesson and locate the folder NYS_PoliceStations
  • Click to open it and make sure that your project file and the csv files for geocoded-police-stations and geocoded-police-stations_rev are both present in the folder. 
  • Go back up a level to where your NYS_PoliceStations folder is located. Right-click on the NYS_PoliceStations folder and choose Send to and select Compressed (zipped) folder.
  • Windows will create a zip file of your project folder with your project file in it along with all the other files that it depends on. This zip file is what you will want to save to your Google Drive or flashdrive in order to access this project later if you are not just using ArcGIS Pro on your personal computer.

If you ran into any issues and need to see how I configured my file the zip file with my project is located below.

Learning Goals

You will sometimes collect location data in the form of addresses rather than as latitude and longitude. In this case, your data will need the extra step of geocoding before you can use it in ArcGIS. Geocoding takes a location like an address or city or state and renders it as latitude and longitude coordinates so that ArcGIS or other mapping programs can read it and mark its location. 

In this exercise, you'll take a list of addresses and use the US Census Bureau's Geocoding service in order to batch process a group of addresses and get their coordinates. You'll read the output file that you receive to determine if the results you got were accurate and to take note of the addresses that the service was unable to provide. You'll find the coordinates that the gecoding service was unable to find matches for by using Google Maps. Finally you'll plot the addresses that you have found using the Display XY Data function in ArcGIS. 

It's important to note, however, that this geocoding service only works with U.S. addresses so if you are looking for addresses in foreign countries, search their government's websites to see if they provide a similar service. Additionally, the site has a limit of 10,000 records so if you need more than that, you'll have to divide up your searches.

Below I've attached the csv file that you'll be using with on the US Census Bureau Geocoding website, it's a list of addresses of locations used in the first two Ghostbusters movies.

Data

Getting Started

  • Download GhostbustersFilmLocations and open it in Notepad or whatever plain-text editor that you use. Notice that it is formatted as a series of lines, with commas between each section. The heading contains the different field names: Location, StreetAddress, City, State, ZIPCode. This is because it is a csv or comma-separated value file, rather an Excel spreadsheet. If you did want to edit this file you could do so in Excel (where you would see columns separating these values rather than commas) but you would need to save it as a CSV for it to work with the Census Geocoder. 
    Remember to keep this in mind for if you want to use this method to get coordinates for your own list of addresses. You can still feed in these values if you are missing a city or zip code for one of your addresses, as long as you leave a blank column (or space between commas) for that value.
  • Close GhostbustersFilmLocations
  • Go to the Census Geocoder online at https://geocoding.geo.census.gov/geocoder/ and choose the option on the left side menu that you'd like to Find Geographies Using... Batch Address Geographies. This will bring you to a page where you can upload your csv file. and the geocoder will use the Census's Master Address File to provide the latitude and longitude for any items that it finds a definite match for or can find a probable match for based on the address ranges available within that locale. 

Uploading your Address File

  • On the Find Batch Address Geographies page go to where it says Choose File, navigate to where you've saved  GhostbustersFilmLocations.csv and select it. 
  • For the options next to Benchmark and Vintage, leave the defaults selected to have it search the most recent Master Address File available. If you knew you were searching for addresses from a much older census and you're worried they may have changed, you could use these dropdowns to select the information from a previous year. 
  • Click on Get Results. You will see the icon in your browser tab turn into the circling line and when it finishes processing the file GeocodeResults.csv will automatically download

Working with Your Results

  • Open GeocodeResults.csv in Excel. Some lines will have Match in column C next to the address and  then what looks like coordinates in column F, Some have No_Match in column C  and no further information.  Addresses that were formerly spread across multiple columns are now only in Column B, and the headers are now towards the bottom of your list in a line by themselves. 

​

  • To know what these other numbers are, you'll want to consult the key available from the FCC explaining what each column is
  • The main columns that you'll want in order to plot your items on the map are
    • A -  The ID (filming location)
    • B - The Address
    • F - Latitude,Longitude.
      However, while you are finalizing the sheet to put it in its final form, you'll still want to know which items had matches and which did not, as well as which matches were not exact so for now you'll want to keep
    • C - Matching Result
    • D - Matching Result - Exact or Not-Exact 
    • E - Address2: which lists the address that the census has determined is the match. This should be the same as column B if the match is correct, but you'll want to have it handy for those rows that are inexact matches to make sure that it hasn't found a false positive
  • Since you aren't looking for anything other than the latitude and longitude for these places, columns G-L can be taken off your sheet. However if you were looking to augment the datapoints with information about the state, county, census tract or census block that contained it, you'd leave these columns. They contain the same code that is used in other census documents for those geographic entities. So if you wanted to look and see if there was anything special about the census tracts that contained the addresses you wanted geocoding for, you'd cross-reference using column 11 (census tract code) to the information about that census tracts on other census documents. For this case, you can delete columns 7-12 (G to L) from your sheet. 
  • Right-click on the first row and choose Insert Row
  • Add in column names as follows: FilmingLocation, Address, Match, Match2, Address2,  Longitude,Latitude (that last one, have the comma in the column name)
    FilmingLocation Address Match Match2 Address2 Longitude,Latitude
  • Highlight Column Longitude,Latitude and click over to Data in the menu options up top in Excel. Click on the icon Text to Columns. 
  • In the Convert Text to Columns Wizard choose the file-type as Delimited and click on Next. 
  • In the next window, make sure that Comma is the only delimiter whose box is checked. Click on Next and then Finish
  • Your Latitude and Longitude are now in separate columns
  • Delete the column towards the bottom of the sheet that contains the headers that originally existed on your sheet. 

The amount of addresses that this geocoder will give you will vary, but at least some of them aren't going to have matches depending on the kind of data you are looking for. The site acknowledges that it is better with residential vs. commercial addresses. However in this case there are only about 3 addresses out of 16 that didn't have matches, so it did have the majority. When the number is that low, it will be easy to just use Google Maps to fill in this information manually. You wouldn't want to do this for a large chunk of your data however as it is time-consuming. You could also try running it again, since I have gotten more matches a second time through than the first, but if this few records are missing matches, it will  take less time to just use Google Maps.

Using Google Maps to find Coordinates for Addresses

You probably use Google Maps quite a bit and didn't realize you were looking at coordinate data. But in fact, whenever you look up an address in Google Maps, the latitude and longitude will be right there for you in the URL. The first number after the @ is the latitude, and the second is the longitude.

  • For each of the addresses that you didn't receive a match, look up the address in Google Maps, and add the latitude and longitude to your GeocodeResults.csv sheet for that address. If there isn't a match for the address, like for Tavern on the Green, do a search for that name and use the coordinates that you find in the URL. 
  • When you've found the longitude and latitude for each address, delete columns Match, Match2 and Address2, and save the file as GecodedResultsForArcMap.csv to a folder you'll be able to navigate to easily. Make sure you are saving it as a csv file, and that there are no spaces in any of the headers. 

You're now set to plot your results in ArcGIS Pro

Plotting Coordinates As X,Y Data in ArcGIS Pro

  • Open ArcGIS Pro, at the Welcome window, click on Map under Blank Templates to start a new project with a blank map under it.
  • The Create a New Project window will pop up asking what your project should be named and the location it will be saved in. Name it GhostbustersFilmLocations and make sure that you have the box next to Create a new folder for this project checked. Take note of the Location it is being saved to because this is where you will find the folder containing all your project files that you will need to zip at the end of this tutorial.
  • Once your project loads up in ArcGIS, navigate to Catalog, by default, ArcGIS opens it in the pane on the right side of your screen. If you don't have it, you can add it from the View menu at the top toolbar.
  • In Catalog, right-click on Folders and choose 'Add Folder Connection'. Maneuver to the folder that you have put GecodedResultsForArcGIS.csv in and click OK. It will now appear below the folder that ArcGIS created for your project, GhostbustersFilmLocations
  • Right-click on GeocodedResultsForArcGIS.CSV and choose Copy. Then scroll up to the GhostbustersFilmLocations folder, right-click on it and choose Paste. Close the folder you created the connection to. 
  • This next part is very important for if you plan on taking your project file and re-opening it on different computers. Do not grab the csv file from the folder you saved it originally and drag it directly onto the map, drag the copy that is in the GhostbustersFilmLocations folder. ArcGIS Pro is very picky about file structure and if you do not keep a consistent file structure, then any layers, files, shapefiles, selections, or any other modifications that rely on being able to access those files, layers, shapefiles, tables, etc, then your project file will be defunct and all those layers will need to be added again, and any joins or other modifications will need to be redone. 
  • Click on GeocodedResultsForArcGIS.csv in the GhostbustersFilmLocations folder and drag it onto your map. It won't appear on your map yet but in the Contents pane, it will appear under Standalone Tables. 

In the previous step, you added Longitude and Latitude information to your list of addresses, and this is sufficient information for ArcGIS Pro to add a point at every coordinate spot you provide that information for.

  • Right-click on GeocodedResultsForArcMap.csv and choose Display XY Data.
  • In the Display XY Data window that appears, go with the suggestions of Longitude as the X Field and Latitude as the Y Field.
  • Change the end of the Output Feature Class  to be GhostbustersFilmLocations but leave the rest of the path exactly the same. 
  • Leave the default in for Coordinate System and choose OK.
  • After it has processed, you'll see that symbols have been added to the map in several locations around New York City, and that a new layer has been added called GhostbustersFilmLocations.

Saving Your Map

  • Click on the Project icon at the top of your menu bar and choose Save. If you are on your own laptop and don't plan on sending yourself a copy of your file to open elsewhere, you're all set. However, if you want to open this project file in another computer, or if you want to send your professor your project because it's finished or because you have a question, this is only part of what you need to do to ensure that you'll be able to access the project later. The other steps will need to occur in Windows Explorer, so exit ArcGIS Pro. 
  • Navigate in Windows Explorer to the path that your project was saved to at the beginning of this lesson and locate the folder GhostbustersFilmLocations
  • Click to open it and make sure that your project file and the csv files for GeocodedResultsForArcGIS are both present in the folder. 
  • Go back up a level to where your GhostbustersFilmLocations folder is located. Right-click on the GhostbustersFilmLocations folder and choose Send to and select Compressed (zipped) folder.
  • Windows will create a zip file of your project folder with your project file in it along with all the other files that it depends on. This zip file is what you will want to save to your Google Drive or flashdrive in order to access this project later if you are not just using ArcGIS Pro on your personal computer.

If you ran into any issues and need to see how I configured my file, the zip file with my project is located below.