Accessing data in QGIS

A few years ago now, I wrote a post covering adding vector layers to QGIS. Much time, development, and improvement has passed since then. Since the usefulness of QGIS increases proportionally with your ability to actually add data to it, it is time to revisit this topic.
QGIS has a toolbar strictly for adding data. It is organized with buttons for each different data source. This seems like a logical way to approach talking about it as well. I will include a brief description of each data source, then the steps for adding said data to your project. I will discuss the following options for adding data to QGIS:

1. Add Vector Layer
2. Add Raster Layer
3. Database Formats
   1. Add SpatialLite Layer
   2. Add PostGIS Layer
4. Web Formats
   1. Add WMS/WMTS or ArcGIS Mapserver Layer
   2. Add WCS Layer
   3. Add WFS or ArcGIS Featureserver Layer
5. Add Delimited Text Layer
6. Add/Edit Virtual Layer
7. New GeoPackage Layer

Add Vector Layer

This dialog used to be the primary way of adding data other than raster, to QGIS. It had, and still has, the functions to add data from a variety of file-based, directory style, and full fledged SQL databases. I think in time, they realized that there were too many different ways of connecting to data, and having them all in one dialog, made it really complicated and confusing. To remedy this, they split out the different data sources to their own icons, and downplayed that functionality in the Add vector layer dialog. What you have left with now, is really a dialog for adding file-based vector layers, instead of more broadly vector layers.
Here are the top 3 file based sources you will be most likely to add:

Shapefile

The shapefile is probably the most widely used format to transfer data. It is a proprietary format created by ESRI for their ArcView product, but has become the defacto standard for sharing data. At this point, it is straight-forward and simple to use. I think its popularity may be on the wane due to other more compact formats being developed, some of which I’ll discuss below, and also the increased availability of layers available as web services.

1. Choose Source Type File. Browse to your desired .shp, and Click Open.
   
2. This will fill in your source dataset. Click Open to load layer into your map.
   

File Geodatabase

The File Geodatabase is Esri’s proprietary, single-user database format. It is file-based, but not a single file. A File GDB is made up of a directory of different files. The directory name is the File GDB name and will end with the .gdb extension. The component files are all encoded, so there is not much use to poking around in them. The format is accessed using the File GDB API from Esri. It allows read, write and basic layer create, though no rasters are supported. An Open File GDB driver has also been developed for read-only access. This process is assuming use of the Esri API.

1. Change the Source Type to "Directory". Under Source choose Type = ESRI FileGDB. Then Browse to choose your dataset.
   
2. Select the .gdb directory itself, not any of the files contained within. Click Select Folder
   
3. This will fill in your Source Dataset Click Open to bring up the list of layers in the File GDB.
   
4. Select the layers you would like to load in the map. Hold CTRL to select multiple layers. Click OK.
   

Geopackage

As I mentioned above with the shapefile, there are new spatial data formats being developed, and gaining a following. The GeoPackage is one of the newest. It was developed by the Open Geospatial Consortium, with the first draft published in August 2015. The specification utilizes an SQLite database as the storage medium, and incorporates spatial standards. It is usable across all platforms and operating systems. You can recognize these files by the .gpkg extension.

1. Leave Source type as "File". Browse to select the GeoPackage you wish to open. You may want to change the File Type to GeoPackage to narrow the list of files being displayed. Click Open
   
2. This will fill in your Source Dataset. Click Open to bring up the list of layers in the GeoPackage.
   
3. Select the layers you would like to load in the map. Hold CTRL to select multiple layers. Click OK.
   

Add Raster Layer

Select any raster layer that is supported by QGIS. This includes any that are supported by the GDAL driver, as well as raster tiles in a GeoPackage, and other specialty formats. You will need to ensure that your raster has a spatial reference defined, or be able to specify one upon loading.

1. Select desired raster layer. Use File Type, if desired, to limit displayed files to those of a particular format. Click Open.
   

Database Formats

Add SpatiaLite Layer

SpatiaLite is an SQLite database with a spatial SQL added in the form of an open-source library. Like the MS Access database, the entire database is contained in a single file. The benefit this has over MS Access is that it is open source, and mostly SQL standards compliant. Like most SQL databases, you must create a connection to a database as the first step to accessing the data within. This process is assuming you have never connected to a Spatialite DB before.

1. Click New Connection.
   
2. Browse to the database that you want to access, and Click Open.
   
3. That database will show up in the Connections window. If you have connected to others in the past, they will also show up in the pull-down. Select the desired database, and Click Connect.
   
4. This will list all the tables, spatial or not, contained in the database. Select the ones you wish to load in. Hold CTRL to select multiple layers. Click Add.
   

Add PostGIS Layer

PostGIS is a spatial extension for the PostgreSQL database. This database is hosted on a server like Microsoft SQL Server or Oracle. The PostGIS extension provides full spatil functionality with both raster and vector data types. This database is used as the backend for many enterprise GIS systems, as well as many other web applications. Like with Spatialite you have to create a connection to these databases, but you are actually connecting to a particular server and specific database name, instead of choosing a file.

1. Click on New to set up a new connection.
   
2. Enter connection information.
   1. Enter a Name for the connection
   2. The Host Name is the computer the database is located on
   3. 5432 is the standard Port. Change if you modified this at the time of installation.
   4. Enter Database Name.
   5. Enter Username and Password for the database. Check Save to store these for later. The password is stored as plain text, so I only store the username. This will require you to enter the password each time you open a QGIS project that references tables from this database.
   6. Check box to Also list tables with no geometry. This will ensure that all available tables are listed.
   7. Click Ok
      
3. Select the desired connection from the pull-down list. Click Connect.
   
4. If you are creating a new connection, you will be logged in as part of that process. If you are using an existing connection at a later date, and did not save the Username and/or Password, you will need to reenter your credentials.
   
5. Once you are logged in, the list of available tables will come up. On the right of the image, is a column for Feature ID. This is a required Unique ID field for display in QGIS. Any table with a Primary Key already defined will have the Feature ID auto selected. If you have created a View, you will need to select the Feature ID field prior to it being possible to select for addition to QGIS. Often, when you click the pulldown to select the correct field, QGIS will make a guess and fill in the pulldown with a name. When they get it right, don’t be fooled, as your work is not done. As you can see in the image, checking the box by your desired column name is what actually marks it. Once you do that, and click on the database name to the left, the Exclamation mark will go away and you’ll be able to select the table. This can be frustrating if you are using a lot of views in your maps.
   
6. Once you have done the above, you will be left with the list of valid tables, both spatial and non, contained in the database. Select the ones you wish to load in. Clicking once will select or deselect as desired. No need to hold CTRL. Click Add.
   

Web Formats

The following are all formats for adding data from the web. These might be shared from a government agency, or data stored in a different database that you don’t have a direct connection to. These layers are similar to databases in that you create a connection to them that is then reusable.

Add WMS/WMTS, or ArcGIS MapServer Layer

A Web Map Service (WMS) is the OGC standard for serving spatial data in a static, tiled format over the internet. Data is being provided more and more often in this format by governments and agencies as a means of providing the public access to data without having to download and maintain it. Since the data is provided in a tiled image format, there is very low overhead for the end user. Most often these layers are available without any login credentials necessary.

1. Create a New Connection. Click New.
   
2. Give your connection a name, then copy in the correct URL. If necessary, enter the Username and Password. You will have to verify with the provider whether they are needed or not. Click Ok.
   
3. This will add the connection to the drop down. If you have others, select your desired Connection. Click Connect.
   
4. This will populate the list of available WMS layers. Select the desired layers to add. Click Add.
   

Add WCS Layer

An OGC Web Coverage Service (WCS) serves spatial-temporal data in a web capable format. This is a relatively new specification, and honestly, one I’m not that familiar with. Check out the Wikipedia article for a more detailed explanation: Web Coverage Service

1. Create a New Connection. Click New.
   
2. Give your connection a name, then copy in the correct URL. If necessary, enter the Username and Password. You will have to verify with the provider whether they are needed or not. Click Ok.
   
3. This will add the connection to the drop down. If you have others, select your desired Connection. Click Connect.
   
4. This will populate the list of available WCS layers. Select the desired layers to add. Click Add.
   

Add WFS Layer

A Web Feature Service (WFS) is the OGC standard for serving feature data in a discrete means. Where the WMS layer serves data in a tiled format, and is static, the WFS layer is actively pulling data from the source. Since it is serving the actual data, it is much easier to overlay this data. As well, these layers may be set up for editing. This format is used to serve data for remote editing purposes.

1. Create a New Connection. Click New.
   
2. Give your connection a name, then copy in the correct URL. If necessary, enter the Username and Password. You will have to verify with the provider whether they are needed or not. Click Ok.
   
3. This will add the connection to the drop down. If you have others, select your desired Connection. Click Connect.
   
4. This will populate the list of available WFS layers. Select the desired layers to add. Click Add.
   

Add Delimited Text Layer

A delimited text layer is just what it means. It can be any form of text file with columns of data, spatial or non. One great thing is that if you have spatial data encoded as WKT (Well Known Text), then you may have all geometry types, not just Point data.

1. Create a layer from a delimited text file. See the basic requirements below the image.
   
   The basic requirements are:
   1. Choose the Text file name.
   2. Specify a Layer name to be used in QGIS.
   3. Choose the File Format. This can be CSV, a Custom Delimiter, or a Regular Expression. That last is pretty awesome as it allows you to take a non-traditional text file, and as long as you can structure the appropriate Regular Expression, turn it into columnar data table. Huge Flexibility!!
   4. Choose Header rows, and Column Names. This allows you to ignore rows at the beginning of a text file without having to manually delete them.
   5. Choose Geometry Type, or No Geometry. If there is geometry, Choose X and Y fields, or WKT field.
2. Basic delimited text settings filled out. Sample data shown in bottom of window. Click Ok.
   
3. Sample data when loaded into QGIS.
   

Add/Edit Virtual Layer

A virtual layer is contained within the QGIS project file. They can be useful for creating small graphics that would not be needed at any time other than the project you are working in. If you are coming from the ArcGIS world, this can be a replacement for part of the graphics toolbar, where you could draw points, lines, and polygons that weren’t part of the table of contents, and were always on top. The benefit to a virtual layer is that you have the flexibility to interact with them the same ways you would a spatial layer from any other source, and they are part of the map layers, so they can be reordered and are not constrained to the top of the stack.

1. The basic requirements are a Layer Name, Unique ID field name, and if a geometry layer, the Geometry Column. Click Ok.
   

New GeoPackage Layer

This is not so much about loading data, but about creating a new layer in a GeoPackage.

1. Create GeoPackage Layer. Basic requirements are listed below.
   
   1. Choose the GeoPackage database. You may pick an existing one or create a new one.
      
   2. Pick a Table name. This will need to abide by constraints of SQLite DB format.
   3. Choose Layer Identifier. This may be a more readable name, as spaces are allowed.
   4. Enter Layer Description.
   5. Name the Feature ID column.
   6. Choose the Geometry type and if applicable, Geometry Column, and Spatial Reference. Always create a Spatial Index!
2. Add Fields for new layer. Specify Field Name, Field Type, and Max Length. Click Add to fields list.
   
3. Basic requirements with a few fields added. Click Ok to create.
   

There are a myriad of data types that can be added to QGIS. This should cover the major ones you will want to use. The web layers all have additional Esri specific types that can be added. The process is basically the same, just a different URL.

For more information about why you should always have a Unique ID field, see this Tip of the Day: Ensure you have a Unique ID for each table in the database

Leave me a comment with your favorite data type!