I'm years out of date, but it competed well with ArcGIS until the expert level, to the point that I gave up my ArcGIS license where I was working and didn't miss much of anything day to day.
QGIS also has server and headless options, plus lots of the QGIS core is importable into python scripts.
I was excited when I learned about a way to create custom georeferenced map PDFs (say, based on an official, but not georeferenced, national park map document). It used qgis, plus one or two other open source tools. It seemed straightforward, if a bit tedious. But I was never able to get the resulting files to work properly on my Android device. The instructions said to use an app called avenza maps, which would understand the georeference data, and show your maps, other maps, plus routes and tracks, all on one display. It just didn't work consistently for me, sadly.
From what I understood, qgis was doing its job properly. It was a neat tool. I guess the whole stack for this use case was just too fragile.
QGIS is the swiss army knife of geospatial computing.
I use it daily, and frequently, for all sorts of work building a location API SaaS company. It's a sterling success of open source, on par with as impactful as Linux, for the geospatial world (IMO).
I tried to perform what I thought were basic tasks of drawing a few line segments and measuring geodesic distances between two points, but I needed a janky plugin for the former and never figured out how to do the latter.
I really do want to learn it however, since it's a lot easier sometimes to work interactively than doing everything in Python code, and I know QGIS is supposed to be a very powerful tool in general.
I learned QGIS arguably the hard way (creating a new non-Earth global basemap from scratch, with no prior GIS experience) but:
- The QGIS Training Manual is good. Not great; don't be afraid to go out of order after the first few modules to get to parts useful to you, because the module organization after "Creating Vector Data" stops being terribly linear. https://docs.qgis.org/3.22/en/docs/training_manual/index.htm...
- https://www.qgistutorials.com/en/ is more project- and task-oriented, and where I learned georeferencing and digitizing (drawing over raster maps)
You can measure distances natively in QGIS, the symbol looks like a ruler. Adding permanent points requires adding a layer, e.g. a shapefile of points.
That being said, as I've outlined in the article, I certainly haven't properly learned it...
But having dabbled in Python/Folium and the like (most recently did geo vis with Superset for another article), some of the datasets and functionalities were familiar (I also had to do point-in-polygon problems in Python, way back when, so maybe that mental scar helped :) ). I found random blog articles and stackexchange posts for specific questions I got stuck on the most helpful.
For instance, I have this link saved: https://gis.stackexchange.com/questions/421467/not-geotagged... , which got me down the rabbit hole of "Oh, you can make a style a function of other things? You can read EXIF? How neat!" (where I then found some other posts).
What I will say, almost everything online is out of date - the GUI changed frequently, plugins are now native functionalities etc. The HTML tooltip thing I did is also a modification of an older blog article (which I'm afraid I didn't save), whereas the contour line styling I took verbatim from the article I linked in that paragraph.
What search terms would one use for finding roles in companies like this, as an experienced dev with a strong background in GIS and open source tools? E.g. Postgis/Pydata/mapbox-gl et. al.
GIS is a huge space, and you could specialize in one niche for years or go more general, so it's really based on what you want to do.
I'd say there's three or four general areas: mapping & map data (mostly focused on visualization), geocoding & search (focused on helping people find things in natural language, think text -> coordinates, text -> point of interest -> coordinate), routing (focused on the mechanics of getting things from point a to point b efficiently), and finally a huge, diverse bucket of "other location APIs" (anything from geofencing to timezones to ETL specialization).
I'm happy to chat about this. We aren't hiring, but there are lots of companies that are.
The moment I graduated and lost access to ArcGIS I got out QGIS and become ridiculously empowered. Yea Arc has Python APIs for some things but QGIS, while rough in the UI department, gives you powerful access to everything. And it plays so well with other things: check out my presentation on using QGIS in robotics (pdf) https://roscon.ros.org/2018/presentations/ROSCon2018_Unleash...
Disclosure: I work for Esri, creator of ArcGIS, so obviously I'm extremely biased. But I'm not in sales, just a lowly dev fwiw.
QGIS is great software, and I respect what they're doing.
ArcGIS Online is a pretty great option and the Living Atlas [1] makes it really easy to consume a lot of data (a bunch of NASA datasets were recently added) and make cool maps. ArcGIS Online ends up being almost like Dropbox for geographic data, but instead of just files you can get full services, and a suite of apps for taking advantage of the services. That includes taking maps offline, making field edits, and syncing them (although that starts to cost money).
You can try the map tool without signing in, and that gives you a good idea of what you can do in the browser (custom popups, charts, tables, filtering, symbology, clustering, cartographic blending, etc). https://www.arcgis.com/apps/mapviewer/index.html?layers=dece...
The personal use subscription is $100/year and includes the full desktop software and cloud storage. [2]
I think you can get very far with ArcGIS Online. There are things you can only do in ArcGIS Pro, but basically everything shown in the article (and a lot more) can be done in the browser. And you get a great set of apps (including Field Maps and QuickCapture) for field editing/data capture.
In my personal experience with using ArcGIS, I have only needed Pro in two situations. The first is when I'm trying to georeference a raster (I just bought one from SkyFi). The second is when I've bought a tutorial book that is based around ArcGIS Pro (e.g. Modern Policing Using ArcGIS Pro).
I won't say more, but I have personally been impressed by the rate of improvement in feature capability in ArcGIS Online. I am not aware of any plans to move Pro to macOS, I feel your pain there.
ArcGIS Pro is just the name for the current iteration of a desktop suite (after ArcMap). ArcGIS Online lacks many of the „classic“ GIS features, especially raster features.
I think it’s just that the web map viewer is a new product and that feature hasn’t been implemented yet (but I’m not on that team and don’t know their roadmap). And it’s not really a paywall thing since the personal use license gets you both online and desktop apps including Pro.
ArcGIS online is absolutely not an alternative to ArcGIS Pro. Not even close. Nobody spending more than a small amount of time using GIS beyond being an end user is going to find online a worthy replacement for Pro. It's a publishing platform with a few analytical tools that each cost credits to run, not a spatial sciences package. Online is good for data capture and integration, but not for someone who wants to perform regular or more complex analytics.
There are currently no plans to rewrite ArcgGIS pro for M1 mac hardware according to ESRI. The move to Pro should have made it much easier to do, but its still a huge task.
The ESRI community have recently had a lot of success with M1/2 hardware, Parallels 17+, Windows 11 arm, and ArcGIS Pro. Just note that you won't be making 3d videos with it comfortably. Everything else seems to be working very well.
I installed ArcGIS "home edition" in Windows 10 for Arm on an M1 via Parallels and it performs rather well. QGis has never really been my thing, I guess I learned Arc (3.2) at too young an age and this old dog wont (no time) learn new tricks.
Their desktop version doesn't. Their target audience is large to very large enterprise. Their customer base is overwhelmingly windows based. They know this, so they've shrunk their dev costs to account for it.
ArcGIS is painfully slow. To apply symbology, you need to go to toolbox, select apply symbology, select the layers, select the symbology file, then make sure to select maintain ranges which for an absolute no sense reason is not the default, then click on apply.
ArcGIS then proceeds to apply symbology one layer at a time, taking between 30 to 75 seconds per layer to apply it.
In QGIS, right click in layer, copy symbology, select other layers right click, paste symbology. Done in 3 seconds.
And like that I could be hours and hours talking about similar experiences.
ArcGIS might be much more powerful, but honestly it isn't worth spending 45 minutes to what in QGIS can be done in 5 minutes.
A lot of people this post are trying to reach don’t know about ArcGIS. As someone who does know about the GIS field, it is frustrating to see post after post about how to do GIS the hard way. The article is fine, but the whole process could have been much easier using the ArcGIS system, and instead of a PDF you’d get full REST services and a suite of apps to work with them.
This post doesn’t mention ArcGIS at all, except for as a data source that your county government probably uses.
I remember when I didn’t know about ArcGIS and thought it was cool to build apps that put points on MapBox maps - which I later found out is a waste of time, because ArcGIS already does that job well.
I think this post leaves people with the impression that GIS is niche, or difficult, or not accessible to lay users. My personal opinion is that recent advancements in Web GIS make the field about as accessible to people as things like word processing, while QGIS is stuck in the ArcMap days, catering exclusively to trained GIS professionals.
It is absolutely meaningful to point out that you don’t need to be a trained data engineer to do this kind of work, and that this 4,000 word journey could have been done in 10 minutes of clicking around user-friendly software.
QGIS and PostGIS were my jam when I worked in that space. We were an ESRI shop so Oracle/SQL Server with SDE (topped with ArcGIS) were the official tools. Some of us were always looking for ways to subvert the culture by building tools based on open source stacks.
One of my favorite experiences from that era: We were meeting with a few ESRI reps for some integration stuff. The lead hot-shot was on his phone playing around during the meeting. He was basically on autopilot. The other two folks were working with GeoJSON response convertors. I said, "I built one of those with TopoJSON". One guy said, "I've never heard of it". I showed them how it was much more efficient and used splines instead of points. The lead dropped his phone and said, "I need you to tell me MORE about that". I showed them the service. They invited me to lunch, I politely declined and said, "today's my last day so I have a ton of things to wrap up". I do miss that realm sometimes.
It's not the software licensing, but the independent consulting entourage that's the problem : internal confidential big four accountancy information told to me without any applicable restrictions on my repeating this put the cost of each Oracle database instance deployed in the F500 at over one hundred million dollars, due to the third party make work.
But it's the other way around. Esri offers so much in the education sector apart from software licenses in a complete package, which you unfortunately have to search for in the FOSS sector.
The business world needs to stop supporting Microsoft Office. The creative world needs to stop supporting Adobe Photoshop. The industrial design world needs to stop supporting Solidworks/AutoCAD. Everyone needs to stop supporting Microsoft Windows/Apple OSX. etc.
I've been telling people for years they should be using GIMP and Linux but nobody cares. At least Oracle's stranglehold seems to be weakening.
Yes, that is the apex of their human capital Ponzi scheme. But, you’d have to wait until the current profs who only know esri retire. Or until they put in the effort to learn how to use foss tools. The only motivation they might have, is a strong belief in foss or pressure from students who know esri is a trap.
That’s not a very good take. Esri is not a Ponzi scheme in any way. There isn’t an alternative for serious GIS. QGIS is a good open source tool but it is nowhere near esri.
As a systems engineer who was hired explicitly for a massive migration from 2 disparate ArcGIS Enterprise environments to an AWS one...
Yeah, there's a massive amount of essential features just missing from OSM and QGIS. Like, even the most basic - layers. They don't even really exist in OSM side of things. Whereas if I'm doing travel-time-bands from medical centers, I absolutely must use multiple boundary layers.
Then again, I'm an engineer, not a data scientist. My colleagues (all data scientists) do all sorts of crazy awesome work. Again, unless you're hacking R/python code and piecemeal it together, you're using ArcGIS. Then again, "ArcGIS Notebook" is a dockerized Jupiter system for doing just that in ArcGIS enterprise.
edit: As for me personally, since I'm working in geoinformatics, I've been teaching myself the hows and whats. Im pretty proficient at both ArcGIS and QGis, as I'm trying to learn them side by side.
If I am doing my own projects, I will use QGis. That's not even a discussion. I'm not about to use proprietary systems to lock up my data.
No, because any further details would highlight where I work and specifically what office.
Lets just say the data intersection of: density and geocoding of users, density and location of office sites, travel time bands, and service level expectations all have a part in the final calculation.
The data is either base layers imported, or calculated layers. There is no one "comprehensive layer" nor does that concept make sense. Different groups handle their own data. We import it in and use their data for our final result.
(In fact, its the OSM approach of everything as 1 layer and tags everywhere is why a DB import and data handling is so obnoxious.)
What features do you think are missing in QGIS which hamper you in your work? I am asking because I am developing a spatial query engine which will work directly with openstreetmap.
I’m not sure that I believe you know what you’re talking about. Not trying to be offensive in that statement, but “can’t back up my very broad, generalized statement with any detail or explanation” comes across as though you’re afraid to reveal your lack of understanding about the details you elided.
You’re saying he doesn’t know what he’s talking about? He is saying you can’t really do GIS without multiple layers. This is basic knowledge for anyone working in GIS.
I am not claiming to be world class expert in GIS and English is not my mother tongue. This comment by tarotuser is confusing me "Yeah, there's a massive amount of essential features just missing from OSM and QGIS. Like, even the most basic - layers. They don't even really exist in OSM side of things. Whereas if I'm doing travel-time-bands from medical centers, I absolutely must use multiple boundary layers." Layers have existed in QGIS for a long time. What are those "massive amount of essential features just missing from OSM and QGIS"?
Explain how qgis doesn’t allow multiple layers. I have a project open right now with 6 layers. That to me seems like “multiple layers.”
And how would that concept even apply to postgis? A postgis query can operate over many tables.
And there are a variety of open source tools that allow multi-dimensional data structures. That’s multiple layers in multiple dimensions.
So I’m hoping, if there’s some aspect of a GIS layer I don’t understand, he can explain it. Qgis totally supports multiple layers in the sense that most people think of it.
He doesn’t have to divulge “secret information” to do this. If he is referring to some aspect of the qgis architecture he can just say so. Qgis is open source, their GitHub repo is clean and well documented. Im merely asking for him to explain why it handling multiple layers is somehow not really handling multiple layers.
What you’re describing sounds like it fits perfectly into PostGiS. You can have tables and you can normalize your geospatial data. With SQL, and temporary tables, you can build absolutely any analytics.
I worked in a GIS Lab with an ESRI endorsed all Arc stuff textbook author/instructor for a number years. Working on similar projects, for anything remotely complicated, I could implement anything he could do, with some python and PostGIS SQL, usually faster, and instantly reproducible.
ArcGIS is a crutch for people who can’t and aren’t willing to program. This goes for all data manipulation tools that aren’t focused on visualization.
If you have an ETL pipeline, probably best to get GIT involved.
They explicitly target the universities as part of their business model. That’s why I say it’s a human capital Ponzi scheme. They give products out to students and educators way below cost, with the specific intention of making the field as a whole dependent on their products. This is well known and pretty explicit. If professors and students had to pay the same costs as private enterprise for esri products the demand for their products would plummet. This is their business strategy and it’s well known and more or less undisputed.
Whether or not they are a good company that provides quality, unparalleled products is not a point I’m contending. They are not competing on that alone. That’s my point.
That author doesn't say Esri is a scourge; more like it's just generally bad to have only 1 provider in a space -- and that it's up to customers to change that by voting with their feet.
I think Esri is (and has been) in a very similar position to Microsoft's in the late 1990s -- having achieved market dominance, they feel like open source software is the biggest threat to their business. But I think the presence of QGIS is creating competition, which is nothing but good for the industry.
ESRI have similar network effect to Microsoft. No individual product is that great but taken together it is very competitive. And ArcGIS online is developing that network effect in a similar way to Office 365.
Also, the GIS space tends to have crossover between different departments in companies. You have a mix of IT departments, GIS specialists outside of IT departments managing infrastructure, data management, casual users etc. Each has a different budget and a different competitive environment. ESRI sits as a compromise between these different groups. Each group may be able to adopt a competitor but not gain traction outside their group.
Not having done much in terms of GIS work, I never had to deal with ESRI until last week.
I was on a call to ask some ESRI rep to add some labeled points to my client's existing map tool. It was a somewhat surreal, weird experience where I got the feeling they were making the work seem much more difficult than it was. Their estimate turned out to insanely off the (my) mark, at eye-watering hourly rates.
At first I thought they had enough business and didn't really care about us. But reading this thread, it seems I was wrong. They are the Oracle of the GIS industry.
In their (possible) defence, stepping outside the the narrow ESRI path of the way you're supposed to do things makes your life a massive pain. I used to work for a company that (among other things) developed tools for clients on top of ArcGIS Online. And I've had many calls where a client asks for a small and very reasonable change, and I embarrassingly have to explain that that is not the 'approved' way doing things in ArcGIS Online, so trying to make that change will essentially require us to rewrite a whole lot of components more or less from scratch and might no even be possible.
I've spent a large number of hours trying to fight ESRI software, lost more fights than I've won and even my victories have mostly been pyrrhic.
Thanks for the perspective, I'm entirely new to the space. We're supposed to use ArcGIS. So if this turns out to be the least painful path, I'll have to learn to deal with it...
ArcGIS the desktop tool, or ArcGIS Online? The desktop tool at least has a pretty complete (but pretty annoying) python API so that if/when you have to, you can circumvent the GUI and automate almost everything. ArcGIS Online is unfortunately extremely inflexible, and things that should be easy are often impossible.
Is there a better option for raw base tilesets? When making my app mapping world tides (https://solunar.pages.dev) I tried OSM for instance but it only had raster tiles at LoDPI resolutions available. ESRI on the other hand had full vector support in a variety of formats^ and provided great docs for integrating with FOSS rendering toolkits. The free tier seems generous, though once it runs out the price increases sharply.
^ For instance, a really neat one that renders ocean features in as much detail as the land typically gets
Mapbox's free tier is an order of magnitude less permissive. They even charge for loading their JavaScript: $5 per kiloload! Feels like the best approach if I'm not trying to commercialize is just to encourage people to self-host using their own API key.
Assuming the service ever even gets to 2M tile requests/month. That alone would be quite the accomplishment.
Though with Gaia going to hell, something new needs to come along.
The cost and effort of deploying something like that in production is decidedly non-trivial. I know because I've done it. Unless you need the customization and flexibility such a solution offers, it's rarely worth it.
Wow, these are gorgeous! Can I ask where you get your DEM data and at what resolution? I’ve been wanting to play around with some relief maps of various bioregions (eg the Great Lakes watershed, Cascadia, etc), but I’ve had trouble figuring out where to find data at the right resolution
Where do we get it?
Only publicly available sources. Usgs has a great portal. Private data is too expensive to get. I was quoted 6 figures for a larger area. They were going to fly a plane and capture it :)
What resolution?
Totally depends on the area the customer would like to cover. If it’s their ranch or property, we usually need 1-meter. If it’s a mountain range than 30-Meter works.
It mainly depends on the resolution limit for 3D printing. So it also depends on the size of the model they want.
Unfortunately not all areas are covered with high res
For the benefit of those who aren't familiar: SRTM was the space shuttle radar tomography mission. 30m resolution is now available (I remember 30m being released for the first time in the 2000s sometime I think).
What are you using to get it into an STL? I've had OK results with DEMto3D plugin, but it has some weird artifacts that I can't seem to get rid of. Are there any better options you're aware of?
I actually use DEMto3D. It's touchy, but I do post-work on the .stl/3d model in blender so it works out ok for me.
If you have weird artifacts, I'm guessing that is due to the underlying data vs QGIS itself. Have you looked at their documentation (https://demto3d.com/en/)?
use these plates on a larger scale to make terra-cotta impressions and turn them into 'chia-pets' such that you can grow micro greens on the landscape of a particular area.
You have fewer products for sale than I was expecting.
I live in NE Los Angeles, which has the Verdugo and San Gabriel mtns, plus Mt Washington and other modest peaks - I think it would look great in this style. Especially because all the development would be excluded.
+1 to this, I can think of a few places id order for but i couldnt figure out how. Did i miss the link? Looks like it refers to "custom orders" but i wasnt sure how that works.
For every new location you do the process looks like:
1. Get the data and prep it for print (fixed)
2. 3D print it (fixed)
3. Rubber Mold (fixed)
4. Wax Model (variable)
5. Bronze (variable)
Steps 1-3 are 40-60% of the costs. So I haven't put the money out of pocket yet to put up new locations. I've let customer's ask first and then do them.
This piece of software brings back fond memories. It was just amazing to discover this after struggling for days with commercial ill fit tools and suddenly feeling i could do anything imaginable. We used this to model and visualize malaria cases and population distribution and plan health care monitoring in liberia. I think the only thing comparable to this feeling is discovering VLC to play video files as a teenager.
I'm trying to find a good project to work on to start learning QGIS, and what you described sounds interesting. Would you happen to have a write-up of this?
sorry not really, but i recommend something where you have an interesting question and at least one dataset you need. A nice starting point is always to get some of your private data exported google and others have about your location history and come up with nice questions. another approach is to browse the plenty of public datasets and use gapminder for inspiration about topics or questions you want to answer.
QGIS is absolutely one of the biggest open source success stories. Its interesting to see the funding and delivery of new features is largely done in Europe where in the US it is extrmely hard to not use ESRI products.
A couple of packages used by qgis worth calling out - postgis which really grew up with qgis, and pdal which is a relatively new but incredible package for working with point cloud data.
Yes, Europe is going the OSS path. I work at University and almost everyone I know in GIS fields switched to QGIS in the past 5 years. Most government departments are also (trying) to switch, some succeeded already (others are waiting for important people to enter pension, so their replacements are more OSS minded).
QGIS is very powerful, but it's not exactly user friendly. The UI is hard to learn, and important functionality is buried several layers deep in non-intuitive drop-down menus and buttons.
I think a simplified GIS program with stripped-down functionality and an intuitive UI could be a big hit. Think of SketchUp versus SolidWorks or ProE.
No argument there.
I've worked with ArcGIS, Autocad, Sketchup, Civil3D, and QGIS. Though clumsy, QGIS is the most powerful by far of these applications.
In many ways that is what we're building at Felt (where I work). We don't offer much in the way of analysis, but we make it very easy (IMO of course) to share your maps.
I think part of the question (and problem) is, which part of QGIS do you focus on? The digitizing and geometry parts? The map-rendering part? Georeferencing? Vector and raster analysis?
Or to put it another way, what's the workflow that a simpler, more opinionated interface would solve, and what features could or would you sacrifice for it?
The most obvious ones. I'm still very new to GIS, but I find it amazing how difficult it is to simply draw some rectangles (of some specified dimensions) and lines on a map, and cleanly label the edges with distances. Annotating, moving, rotating things... is quite a pain compared to modern UIs. So I wonder if there's a slim and beginner-friendly open source version of QGIS.
"Most obvious" is still pretty subjective, though. It depends on how you wind up looking for a GIS tool.
- A lot of people come to it wanting to edit or annotate existing city/street maps with geometry, like you're describing.
- My first encounters with it were trying to plot points on a map to build a heat map, in a data analysis context, and I was coming from Google Earth/KML.
- The next time, I wanted to georeference raster maps to make more accurate vector data, and I was coming from Inkscape and Illustrator.
- Another popular use case is to work with topographical and height data, which is another workflow with mostly another interface in QGIS.
- And all of these can have very different output goals - slippy maps with raster or vector tiles, topo maps, 3D renderings and prints, GeoJSON and PostGIS data.
The "most obvious" app for each of those use cases would each be different, with different interfaces. And I can see where that would be good for beginners, or for limited use cases, but at some point each of those use cases intersects with another. Drawing bike lanes on a map quickly means working with topo data. Georeferencing a new base map eventually hits plotting POIs and drawing things like political borders as geometry. Outputting a heat map from data you plot in QGIS can work, but then you need to plug it into a database at some point so the data can be updated in a shared or automated context.
IMO what would be more useful for QGIS is a ground-up, no-assumptions review of its UI similar to what Blender did in 2.8/3.0. Find where they can consolidate redundant or conflicting tools and interfaces, redesign their mouse/keyboard interactions and tool palettes around how users intuit they should work, and provide alternate modes for the more complex workflows that existing users are accustomed to. Blender's redesign proved pretty well that an esoteric UI can be streamlined and modernized without sacrificing power user workflows.
But I don't know if the QGIS UI community is as large, organized, or focused on modern UI design as Blender's, which was a rather unique case in FLOSS. And it'd be difficult to divorce QGIS's GIS data-editing engine from its interface as a whole to build standalone tools with smaller UIs focused on tasks instead of toolboxes.
Plugins might be able to do this, which could be an interesting approach. Instead of adding features like plugins tend to do, one could ship a "flavor" of QGIS with pre-installed and -configured plugins that provide narrower, task-focused interfaces around existing features.
Some years ago QGIS was useful to me in spite of its UI: I got a PDF map of certain infrastructure that was not georeferenced, and I was able to correct its projection to Web Mercator with great acccuracy. But at the end I was somehow unable to find out how to export the data! QGIS did show me with the transformation matrix it had built and I wrote a program that applied that matrix to my files.
In the vast majority of cases you cannot turn a PDF into spatial data, except for georeferencing it as an image. There are some exceptions, such as when the PDF already includes georeferenced vector data, but its very unlikely you had that sort of file given the issue you described.
You likely either:
- needed to export an image of the data (raster);
- needed to digitise the data within;
- needed to extract the vector data first from the pdf and work out how to deal with it.
Opening a pdf without appropriate data structures into a gis software package is akin to taking a picture of a billboard that has a printed image of a map on it and then expecting to be able to do anything with that picture. The pdf is a bit better, but not by much unless it began life as a georeferenced pdf with vector data maintained.
The PDF was a map of infrastructure that was likely produced with ArcGIS.
By the time I got it, it was just a vector image.
The infrastructure in question was pipelines, represented in the PDF by lines, so georeferencing them was enough to get spatial data adequate for the application.
I used to program python plugins for QGIS, I really miss it. It made me feel really useful, I'd write visualization algorithms one day and the next it was deployed to the whole company and people would email me telling me how much easier it made their job. I don't think I've ever felt this valued since, every other job I had was a small part of a big whole.
Hello, would love to chat with you about QGIS plugins. Email me at devjobs@anor.io if you would be interested in making a QGIS plugin for the solar PV industry :)
Ideally a conformal mapping projection with minimum scale error.
I'm saddened that complex-number polynomial expansion fell out of vogue and fear that was the last wall that kept the muggles out of hard core map projections.
I love exploring open GIS datasets with QGIS. It's incredible what this tool can do and I keep discovering new features or plugins all the time, which keeps it interesting. The amount of publicly available GIS data combined with tools like QGIS and PostGIS can be a big time saver. I don't work in the space, but occasionally run into problems requiring some form of support for geospatial data. Need a list of US cities and postal codes along with their respective coordinates? Easy. Find the dataset, import into QGIS, export to PostGIS. Done.
I spent a decade in the GIS space, ending back around the final days of ArcGIS 9.x, or very early days of 10.x. I also remember various alternative and early open source efforts of the time.
Then, about a year ago, after several years away from the scene, I got some mojo back and started investigating modern QGIS. I was very pleasantly surprised, it immediately felt like an upgrade from what I used to use. And also felt like coming home, in that it was obvious and intuitive how to accomplish the standard functions. If someone had put me in (that famous thing) a blindfolded time machine for a zero-year trip, popped me out, put me in front of QGIS and said "Welcome! This is ArcGIS 9.6!", it wouldn't have been at all unbelievable. If you know what I mean.
No it's not perfect. Casually and hastily wing your way around a project, copypasting here and there, setting up all kinds of parameters and advanced display bits&bobs, and - like most complex geospatial software - you'll, once in awhile, find yourself unceremoniously dumped out onto the desktop with a software error. So a highly robust and restore-able workflow is still essential. But regardless it's up there with the very best packages I've ever used.
Interesting, I have had the opposite experience to that. I spend a lot of time in ArcGIS late 9.x. and early 10.x and QGIS is not even remotely intuitive to me. You can drop me in ArcGIS Desktop or Pro years later and I can still intuit how to do everything, but I have to Google how to do anything in QGIS.
If you can, get yourself in front of an up to date version of ArcGIS Pro for a little while. The user interface will feel entirely different, and some people don't like it because it's different, but the entire package feels significantly better.
Importantly, a good deal, though not all, of the infuriatingly frustrating "random esri error for who knows what reason" issues have been resolved, and continue to improve.
I’m slowly making a map of my back yard in QGIS. One annoyance: I live in a place where tectonic movement over my expected lifespan is significant on the scale of my map. And so are the supposed inaccuracies in various coordinate systems. (I want to record where things are so I can find them again without digging big holes!)
As far as I can tell, QGIS has no particular understanding of either a coordinate relative to the (moving!) crust or of a coordinate in space-time that can be projected to space at future or past times. Surely this should be a thing!
I found HTDP, a web tool that can shift coordinates forward and back in time:
(I have an RTK-capable GPS and NTRIP data via UNAVCO from a nearby CORS station. I was hoping that storing position relative to such-and-such CORS station as it was on such—and-such date would be straightforwardly doable in QGIS.)
Wow that’s a pretty interesting (and scary) challenge.
I know that the coordinate systems get revised every few years. In Australia we used to use AGD83 then it was revised to GDA94 and more recently to GDA2020 — and this was to account for tectonic shifts. And there’s calculations for transforming from one to the other.
That’s the only clues I have on that side of things and I’ve forgotten most of what I’ve known.
Also there’s a concept called “rubber sheeting” for transforming points where you know the amount of error at a bunch of points and you want to transform all the other points by interpolating how much error there would be at those points. The might be useful too, if you’re implementing a solution yourself.
I didn’t find anything in QGIS to give a layer a CRS and an associated date, so I’m not sure how coordinate systems like this are expected to work properly.
I am doing the same, Mainly I wanted to record where pipes and such are located but might as well record buildings, fences and trees while there. I was not worried about geologic movement but all the global coordinate systems felt very clunky when locally mapping at the centimeter level, I finally settled on having a well defined surveyed reference point, and using meters north and east off that.
Perhaps that would work for you. because while the plate is shifting with respect to the globe, everything on that plate will maintain the same relative position.
I set up my own NTRIP base station at a fixed point in the middle of my roof with a hefty bracket, did a PPP survey to determine its location once, and am considering everything relative to that. This spring when I'm back at the surveying, I'll likely do another PPP run and make sure it hasn't moved too much. If it has, I guess I'll have to figure out how to reconcile that. From what I've gathered, I don't think this is too far off from how real surveying works.
I didn't see much reason to use a different unit than degrees. Although while I haven't gotten super in depth learning QGIS, it feels like there's an impedance mismatch in that it seems to be a 2D program that treats degrees as linear units (and then applies a fudge factor to degrees longitude), rather than a native 3D program. So I'm doing all of my collection, point storage, and calculation with scripts outside of QGIS, and then only pushing the cooked results to QGIS for visualization.
I don’t have a good sense for how far one needs to be from one’s NTRIP base station, but I’m pretty happy with the UNAVCO data I’m getting. The nearest station is operated by Serious Professionals, and it won’t move if I mess around with my roof. :)
I've got a professional base station that's decently close, but still far enough to degrade the accuracy a bit (I forget the exact figure but you can find it if you search). When I used the professional base station to figure out the point and compared it to what PPP had found, it seemed to be within that amount of error.
I chose the part of the roof least likely to be disrupted. But if I do need to move it, then I'll have to figure out exactly how I want to map to points going forward.
Instead of a reference + north/east meters, how about two reference points, and then everything is referenced as polar coordinates from there, point A to point B being 0 degrees. My concern would be if the direction of north/east changed over time from the plate movement.
Oh very, very much yes. I've been using QGIS for about five years now as the first step in doing any sort of visualization work around GIS data. Better to trust a GIS application than my own bad d3 code :). It's also been absolutely fantastic for changing projections, changing resolutions, merging GIS files, things like that.
My only complaint about it is that it can be a little bit slow sometimes. It has to redraw everything if you zoom in. That's especially annoying if you're working with data that has millions of multilines or points. But, as a prototyping engine, that's kind of expected and still better than paying for ArcGIS.
Our project is not big enough to justify tools like ArcGIS. Previously I'd used Global Mapper, which is also commercial but is somewhat more reasonably priced.
But I recently gave QGIS a shot and am very happy with what it can do. And the price is right!
I used QGIS to help me import some terrain into Minecraft. I found a lidar elevation map, used QGIS to visualize it in 3d and tweak some things, and then I was able to use another tool to do the conversion into Minecraft. Definitely an impressive tool.
Every field has its own “occupational hazards” - I found that with maps and mapping the maps themselves were so fascinating, you could get drawn in and lose a lot of time just… exploring I guess.
QGis is great, lots of tools are nice. Generally ESRI sucks up all the oxygen and leaves not enough for innovation and implementations from anyone else. There are a lot of features and capabilities needed in the Geospatial world that do not have reliable, rock-solid open source implementations. (I could’ve provided examples a few years ago but now I’ve forgotten the important details and only have opinions left)
I've been using QGIS a bit for simple things. Viewing OSM road data in a PostGIS database. Viewing geojson files. Everything has been more difficult than I expected, but I think that might just come with the territory. I'm working through PostGIS in Action to try to get the requisite background knowledge.
One thing that's frustrated me, and I want to make sure it isn't a misconfiguration on my end, is that QGIS feels really slow. For example, I have a 150MB geojson file which has 300,000 points with associated metadata. Even when I'm zoomed in such that I can only see a few thousand points at a time, if I pan the map over by 50% it takes at least 10 seconds before it loads in the new points. Many long operations seem to take place synchronously on the UI thread, so the whole app is unresponsive while they take place. Clicking the drop down arrow on a PostGIS Schema to view the tables spins for several minutes. No other Postgres tool I have takes that long, so it's not the database. The PostGIS import/export tools were also extremely slow and didn't have progress bars. I'm using 3.24 currently. I don't want to rag on it too hard, but it's really hampered my enthusiasm for working with maps and GIS.
You'd probably be well-served by exporting to some format with a spatial index. GeoJson is just a textfile, not much better than reading in a CSV. The current recommended format is Geopackage (gpkg), which is based on SQLite.
Edit: agreed on the slowness of DB connections, I've found that too, including for relatively small locally-hosted DBs.
Yeah, converting it to gpkg sped it up a lot, thanks!
I assumed that since geojson is completely unsuitable for querying directly, it would load the whole thing into a native in-memory format, but perhaps not.
I don't think geojson is a great format for anything with more than a few MB of data.
I wanted to see exactly how bad it is with a largeish datasets, so I exported the New Zealand address dataset[1] with ~2.5M points as a geopackage (750MB) QGIS loads this fine its a little slow when viewing the entire country but when zoomed into city level it is almost instant to pan around.
Using ogr2ogr I converted it to ndgeojson (2.5GB), It crashed my QGIS while trying to load it. Using shuf I created a random 100,000 points geojson (~110MB) it was unbearably slow in QGIS while panning around 5+ seconds.
I currently use and recommend flatgeobuf[2] for most of my working datasets as it is super quick and doesn't need sqlite to read (eg in a browser).
It is also super easy to convert to/from with ogr2ogr
QGIS is always a little more difficult than expected. Even as someone with a masters in GIS, its still hard for me to remember where they put things or how to do it. Its extremely powerful but can be extremely frustrating too.
GeoJSON is unfortunately one of the worst formats to keep the data you are using in. Its great for transport and interoperability, but there is no way to index the data in the format. It doesn't matter if you are only drawing a few thousand points, it is still looking through all of them to see which ones to draw.
GIS is awesome, and don't let one tool get you down. If its something interesting to you, there is a lot more to it than just QGIS. Part of my pain with QGIS is because I rarely ever use it so I forget what I learned last time. I spend most of my time purely in python, and don't really need to.
No, you're not victim of misconfiguration. QGIS is very slow with large datasets, no matter what source you use (even if you have Oracle with costly Spatial extensions). It is due to some unfortunate design decisions - the app always loads the full dataset from the source, and draws/shaws every single feature. It is possible (through some recent, obscure check in the prefs) to tell it to skip smaller objects when putting everything on the screen, but generally is not possible to tell it "do not load features from PostGIS that are going to render to only few pixels".
Another reason is that the underlying C++ engine is tied in such way with the fairly-large Python codebase, that it seldom uses all CPU cores. so you have a single process app. Add to this the fact that the GPU is not used extensively and you get one very slow APP.
So, yeah, for datasets larger than 50k points its slow on arbitrary hardware. Some analysis are impossible to run unless you go to PostGIS
To be honest, ESRI is not a company that everyone loves, but they really invest into ArcGIS Pro, even though they also are not there yet. Both QGIS and ESRI shy away from spatial SQL which is like times more effective for spatial analysis.
To be honest, QGIS is quite old-school in design, and the core devs know that it would take enormous effort to implement it from scratch... at least the slow parts. The fact that it works and is open source, does the fundamental things you need does not make it the _top_ software. this "opensource ftw" is really stupid when you have to work with hundreds of layers with hundred Ks of points.
Make sure you have estimated table metadata turned on, otherwise QGIS will run a bunch of queries to understand your tables I believe.
We typically use QGIS as a viewing engine only, if you let postgis do the heavy lifting it's a beautiful setup, especially with a tuned dB and a indexed clustered postgres table
> Make sure you have estimated table metadata turned on, otherwise QGIS will run a bunch of queries to understand your tables I believe.
That fixed it, thanks! Looking at the manual, there is a tip that tells you to turn that on, otherwise it will read the entire table to characterize the geometries... It's a bit mad that the default behavior has it querying potentially gigabytes of information across the network every time you open the app and click a dropdown, with no progress bar. But it's definitely the type of app where you need to read the manual, and it says so right there.
A couple of years ago I started working with drones and writing a bunch of custom software. One of my pet peeves in the drone industry is that a ton of stuff uses “reinvented wheels” that are drone-specific or product-specific instead of just riffing on open standards. Given that, I decided to spend a day or two and do a bit of a survey of the different open standards that are available around kind of the GIS/mapping world, the aviation world, etc.
At this point I didn’t even know about QGIS, but I was happy that instead of reinventing the wheel I already had well-defined file formats etc. to work with. QGIS came a little bit later when I had a bunch of data to analyze. My image metadata wasn’t exactly right for QGIS (and couldn’t have been done that way in real-time anyway), but a 3-line shell script using GDAL converted all of my images into something QGIS could load and draw directly overlaid on top of Google Maps. My flight plans were all done in annotated GeoJSON, which QGIS happily loaded as-is and helpfully split it into the layers I had used under the hood as my data encoding. There were a couple of files that were just plain-old CSV, and QGIS happily ingested those and drew them as points and lines and line segments as appropriate.
It’s phenomenally good. Some other people in this discussion have mentioned that the UI is a bit rough and I can’t disagree with that, although I feel the same way about the commercial tools too. It’s a power tool for sure, but it doesn’t take too long to figure out the quirks and build yourself a workflow.
Don't forget, QGIS is one of several projects of OSGeo. A significant part of the fundamentals come from GDAL, another OSGeo project, which is very excellent if you ever need to do some command-line processing of or need a library for raster or vector GIS data.
Their licensing isn't perfect - the QGIS installer, for example, more-or-less requires acceptance of a few free-as-in-beer but still proprietary licenses, which are used for some specific raster formats. Still, it's one of the great free software success stories, I think.
It definitely feels like they will eventually displace ESRI. Most shops will probably keep ESRI around for some format conversions or niche tasks. However, the day-to-day desktop GIS is almost exclusively QGIS in the places I've worked, if only because you can throw it on any machine you want to without worrying about licensing.
I did call out GDAL, but missed the whole OSGeo ecosystem, I’m afraid.
Very cool to hear that QGIS is such a prominent piece of software in professional settings! I’m just a software/data engineer who dabbles in this stuff for fun, so I had no idea. Lots of comments here asserting the same thing.
It would be amazing to see OSS (even though not truly FOSS) replace industry giants. I see this happening a lot in my space, where a lot of OSS (mostly Apache, some MIT and consorts) overtakes commercial offerings.
Excellent article, by the way, it's very useful to have this kind of writeup.
I suspect it's something to do with the distributed installation, and the amount of the required behaviour that is sitting behind completed specifications. Eventually, it will be cheaper for a user to just pay for development of any remaining features they need than pay for licensing. And, the functionality addition is cumulative - pay once, keep forever.
I mean I don’t want yo dis ping your experience but no shot. I’m not sure where you worked but esri just is a massively dominant player still. I’m government and infrastructure, which is esri’s primary market, QGIS is nowhere near replacing it on the desktop. Now, QGIS is a great project but we can’t be naive about the situation.
QGIS is fantastic and that team deserves a lot of praise for their work over the years. However, the article calls out using KML as a data source and unfortunately QGIS does not have the best support for KML. One of the reasons KML is still around despite it's faults is because a KML author can easily design a UI for their data in Google Earth, organizing data by folders with descriptions, including pre-defined views and image overlays among other things (a basic example here: https://kmzview.com/5qBGblENff4w0RkQf89J). Then, anybody who opens that KML in Google Earth gets that same UI. There are a lot of casual consumers of this type of data and this experience is powerful for them. I think QGIS could really broaden their appeal to this casual user base by supporting these well-loved KML features.
Shapefiles are no more proper than KML, I'd even say KML is a better format, since it's a single file and doesn't have the archaic name limitations of SHP.
KMLs do not have a spatial index. Reading a properly constructed shapefile with 2m points is fine, and running a SQL query on it is also fine. Trying to do that with an xml based file is not fine. Each of the major GIS packages spends quite a bit of effort to convert kml to an in memory spatially indexed dataset on ingestion. Shapefile conversely does not have to sit entirely in memory to still be reasonably quick.
Shapefiles do have the anyone field name length limit, among a few other things, but alternate formats that recognise that issue have existed and been popular for well over 20 years.
In other news, geopackage is based upon sqllite, supports long field names, a good assortment of data types, and is finding good adoption among many facets of spatial products from desktop to scripting, etc.
use PostGIS with proper spatial indices, and properly tuned if you have to take the QGIS way. and remember that spatial SQL is always better than using python-based analysis. particularly with newer versions of Postgre where there is chance that the queries be parallelized.
QGIS is great, I use it daily and I hugely prefer it (and it's mobile equivalent Qfield) to ArcGIS and FieldMaps.
The educational world is pretty split between ArcGIS and QGIS. Students don't want to pay over $100 for a yearly ArGIS student license, but more advanced geostatistical analysis isn't supported yet on QGIS. Progress is slower in industry, especially in larger companies. Other critical software like Autodesk, Vulcan, DESWICK, MODFLOW, and Leapfrog already work (somewhat) smoothly with ArcGIS.
QGIS is just another thing to go wrong in managers minds, and there is zero opensource progress in developing applications as powerful as QGIS in geomatics adjacent fields.
Can you provide a few examples please for your statement: "but more advanced geostatistical analysis isn't supported yet on QGIS"? I'd like to hear more about this.
I'd rather use simplified command line things instead.
For example I don't even remember how to generate raster tiles if I want to host my own maps, I don't think openstreetmap distribute the software they use to make their tiles, out they don't really document how to do it.
So like most open source initiatives, it's not all good.
QGIS isn’t bloated, it’s a big tool for a big problem space. You can’t use “simplified command line things” for a vast majority of GIS tasks, if only because they require visual interactivity.
That having been said, why on earth would you want to use a GUI application to generate raster tiles? The only input required is an extent and a zoom level. Use GDAL or any of ten other free and open-source command-line tools readily available.
It sounds like you’re leaving a dismissive drive-by comment rooted largely in your misunderstanding of a large and complex field with use cases which are largely disjoint. Analysts need GUIs and a lot of different analytics tools. People making web maps don’t.
I'm quite late to the thread, but has anyone found out a way to use QGIS programmatically?
For research you often want to write routines that you can re-run in the future (replicability, reproductibility, etc.) and it's very easy to do so with statistical software (Stata, R, even Py) but with QGIS I just have a README saying "click here, click there, etc.". Tried to wrap everything in a QGIS Python script once, but it was impossible as there was not a one-to-one translation between clicks and code.
It wasn't 1-1 with clicks, and it took a ton of trial and error, but I managed to get there eventually with a project. Here's the passable write up I made for it, theres a link to the final script towards the end, hopefully theres enough to be helpful.
A couple of weeks ago I got a point cloud file with no clue about how to open it. Sure enough QGIS came to rescue and I learned it has a 3D view as well. I'm grateful for it :)
reply