Web Servers and Image Servers



Manifold has the ability to bring data into a project from a seemingly endless range of different types of data sources, including almost any file format or database management system.


In addition, Manifold also has the ability to reach out through Internet to fetch spatial data, both vector drawings and raster images, from web servers that provide such data using a variety of well-known protocols.   


Such web servers typically provide data dynamically to fill in whatever view we are looking at in a window that shows content from the web server.  As we pan or zoom in or out in the view Manifold automatically tells the web server what view we are looking at and the web server responds by sending the data needed to fill in the view.  


This topic discusses web servers and image servers.  Image servers are a very popular form of web servers which use their own protocols and have become so famous, such as Google Maps, that they are worth supporting with pre-configured modules.  Image servers are often referred to as imageservers.   The illustrations in this topic show examples of different web servers and image servers.




To create an imageserver data source:


  1. Launch Manifold.

  2. Choose File - Create - New Data Source.

  3. In the Type box choose Web Server: imageserver.

  4. In the box just below the Type box choose the desired imageserver, for example, Bing Maps Street Map.

  5. Press OK.

  6. The new data source appears in the project.  Click on the + icon to expand it.

  7. Double-click on the image within to open it and see what the imageserver provides.


For examples, see the Example: An Imageserver Tutorial topic or the Example: Create a New Data Source from a Manifold Image Server topic.




To create a web server data source:


  1. Launch Manifold.

  2. Choose File - Create - New Data Source.

  3. In the Type box choose the desired web server technology, for example, Web Server: wms.

  4. In the Source box enter the connection string / URL required for the desired server.

  5. Press OK.

  6. The new data source appears in the project.  Click on the + icon to expand it.

  7. Double-click on the image within to open it and see what the we server provides.


For an example, see the Example: Style Applied to an Image Server Image topic.

Web Server Data Sources

Web servers normally send either raster data such as images or they can send vector data that represent objects in drawings.  When they serve raster data such data normally is sent in the form of tiles, rectangular groupings of pixels that are 128 x 128 pixels or 256 x 256 pixels or some other convenient size.   Sending tiles through the web is much more efficient either than sending individual pixels or computing on the fly and sending a screen's worth of pixels.   For that reason such web servers are often called tile servers.


Besides the obvious use of pixels and imagery in the form of tiles within web servers that provide imagery such as overhead satellite photos, as, for example, Google or Bing Satellite view do, web servers will often also use imagery to provide a visual representation of fundamentally vector data as well.   


For example, street maps that show roads are maintained on the servers as drawings, that is, vector data.   To show those street maps on the web, the server creates an image, like a screen shot, of the vector data complete with all formatting, labels and other accessory graphics arts wonderfulness that helps create an appealing map.   The server then sends that imagery in the form of tiles out over the web.  



Sending a visual representation of vector data out over the web in the form of imagery, that is, screenshots of the data, is a good thing for people who just want to look at the map.  They can look at the map using ordinary web browsers or simple software on their telephones, tablets or computers that need not do anything more complicated than put together images from tiles that consist of well-known, simple image formats like jpg or png.   


But that is not a good thing for people who want access to the actual vector data so they can use it for GIS purposes.   For example, if we want the actual vector line objects that make up the boundaries of real estate parcels so we can accurately calculate the areas inside an image where the boundary lines are just fuzzy swaths of different-colored pixels (as they are in an image) is not the data we need.    For such purposes there are protocols, such as WFS, that can be used by web servers to provide the actual vector data served dynamically.il_sphinx_sm.png

Image Servers

Manifold Image Servers, often written in concatenated form as imageservers, are a subset of web servers that serve imagery in a form supported by dedicated modules specific to each such image server.   Such image servers typically do not utilize an "open" protocol but use their own specific protocols and methods to do something similar, serving tiles dynamically on demand to fill in a desired view.


Manifold image server modules are snippets of code which use the openly published Manifold Image Server Interface (ISI) to automatically fetch image tiles from popular image servers like Bing, Virtual Earth, Wikimapia, Yahoo!, Google Maps, Yandex and many others. Image servers can provide street maps, overhead satellite imagery, combinations of streets and satellite imagery and other data as well.   


Anybody can write an image server module using the ISI interface and many such modules have been written by users and published for others to use.  


Many image server modules are already built into Manifold.   Installing an additional, user-written image server module is easy: simply place the DLL for the image server module within the Manifold installation path, typically in the ~/Shared/Bin folder.   When Manifold or some other Manifold product that supports ISI launches it will automatically utilize that DLL to include the provided image server module.il_eiffel_sm.png

Connecting to Web Servers

Just like we must know the URL of a web site to connect to it, we usually must know the protocol utilized and the URL to the web server which provides the data we would like to use.


Some types of web servers, for example well-known image servers such as Google maps or Bing maps, are built into Manifold in the sense that their default connection strings are pre-loaded into the dialogs that invoke them.   Adding dynamic imagery from such servers is as simple as choosing the desired image server from a list in a dialog.    Other types of web servers, for example those utilizing wms protocol are more general and connect to a vast variety of different servers operated by various web sites, individuals and institutions.  To use those we must note the connection string (a form of URL) required and enter it into the dialog.



Some web servers are free to use without any restrictions at all.   Some utilize access controls such as the need to enter a special key or to incorporate login credentials as part of the connection string.   Manifold dialogs assume the user is capable of visiting the desired web site to learn about any restrictions or special credentials required to access a given server.


Some web servers, often those which use WFS or similar to provide vector data, can be astonishingly slow.  If no data appears when using such servers, wait for a minute or two and check the Log Window to see what is going on.






The New Data Source dialog provides the option to Cache data when using a web server source.   




Using cache can provide faster performance with slow web servers because data that is reused, say, because we return to a previously-viewed scene, need not be re-fetched.  If we are no longer connected to Internet or if the remote server goes down, the data source can still show what tiles have been downloaded into cache.    


Cache data for image servers is stored in cache tables within the Cache folder in the System Data hierarchy.    If cached data is no longer desired, individual cache tables or even the entire Cache folder can be deleted to reduce the size of the project file.  When the data source created with the cache option enabled is opened again, the Cache folder and cache table for that source will be re-created so the cache can grow again.


il_webserver_cache01_03.png  il_webserver_cache01_04.png  il_webserver_cache01_05.png


The downside to using cache is that cache tables can accumulate many megabytes of data within the project file.   Beginners should uncheck the Cache data box since it is easy to forget that caching data and then browsing can grow apparently small projects to large size.   See the discussion and illustrations in the File - Create - New Data Source topic.



Problems connecting - Check the Log Window to see what is going on behind the scenes if an attempted connection does not work.     The problem is usually a wrong connection string, failure to provide required credentials such as an key string that authorizes access, wrong choice of protocol (the server uses WMS and the user picks something else), an incredibly slow server, a server that is offline or a server that is wrongly configured and which is not correctly using the protocol it claims to use.  Visit the Manifold community forum and talk out difficulties with other users.    The very last thing to consider is the possibility that a bug in Manifold is causing the problem.  Bugs happen, of course, but leaping to the conclusion as a first step that a bug is the problem is almost always a mistake.  tech_ravi_sm.png


Tech Tip:  Some web servers to which we might connect using WMS, WMTS, WFS or other connection standards can be incredibly unresponsive or, despite allowing a connection, will not allow data to be fetched.   If pressing the Test button results in a successful connection, or if layers available are displayed when we expand the data source, but then when we try to display an advertised layer nothing happens, we can open the Log Window to see what is going on.


Here are a few lines of log window information for a happy session, connecting to a WMS server in New Zealand run by the ever-efficient LINZ organization:


Web request: (root)::[Data Source] (GetCapabilities) (0.578 sec, 13.0 KB)

Web request: (root)::[Data Source]::[layer-767] (GetMap) (1.255 sec, 47.9 KB)

Render: [Data Source]::[layer-767 Image] (1.561 sec @1.0)


In contrast, a few lines of log window information for an unhappy session, attempting to connect to the WMTS server run by the French cartographic service IGN but getting no image for a cadastral parcels layer:


Web request: (root)::[IGN]::[CADASTRALPARCELS.PARCELS_bdparcellaire_b] (GetTile) (0.070 sec, 0 B)

*** (root)::[IGN]::[CADASTRALPARCELS.PARCELS_bdparcellaire_b] (GetTile) The remote server returned an error: (403) Forbidden.

Render: [IGN]::[CADASTRALPARCELS.PARCELS_bdparcellaire_b Image] (0.315 sec @1.0)


From the above we see an empty cadastral parcels image was rendered because the remote server refused to provide tiles in response to the GetTile request, returning a 403 error Forbidden.    Finding out why a web server refuses service can involve some detective work.  


The server might require a special key for access, a key we are trying to use may be out of date, a login and password may be required for some resources, the connection string may have been copied and pasted inaccurately or, of course, in the case of IGN the server may be refusing to work to show solidarity with striking railroad workers or air traffic controllers.


Keep in mind that just because a web server offers tiles via various standards like WMTS or whatever does not mean that it will be fast or even reasonably tolerable.  Some web servers are painfully, incredibly slow and will take many seconds or even minutes per tile.   When images do not appear or fill in with painful slowness a look at the Log Window can show the problem is not our Internet connection or Manifold, but just incredibly slow response from the web server.

See Also

File - Create - New Data Source


Example: Spectacular Images and Data from Web Servers - A must see topic providing a gallery of views illustrating how Manifold can use web servers such as imageservers and other free resources to provide a seemingly endless selection of spectacular background maps, satellite images and GIS data with nearly zero effort.


Example: An Imageserver Tutorial - An extensive tutorial showing step by step how to add new data sources that are imageservers, how to show them as layers in a map, how to create a new drawing that matches the projection of the map and how to trace over what is seen in an imageserver layer to create an area object in the drawing.


Example: Create a New Data Source from a Manifold Image Server - Manifold image server modules are snippets of code which use the Manifold Image Server Interface (ISI) to automatically fetch image tiles from popular image servers like Virtual Earth, Wikimapia, Yahoo!, Google Maps, Yandex and many others. Image servers can provide street maps, overhead satellite imagery, combinations of streets and satellite imagery and other data as well.  Using Manifold Image Servers is one of the most popular Manifold features.


Example: Create a New Data Source from a MAP File - Create a new data source from an existing Manifold .map project file.   This is the classic way to nest projects, one calling another, to create libraries of data and projects.   Access to nested projects has effectively zero performance loss and the links within projects take up effectively zero space so we can create huge constellations of data at our fingertips.


Example: Create a Data Source within an Existing Data Source - When a data source is writable, for example, if the data source is a Manifold .map file, we can work within that data source as if it were at the top level of our project.   For example, we can create a new data source that is nested within the existing data source.   This example shows how.


Example: Create and Use New Data Source using an MDB Database - This example Illustrates the step-by-step creation of a new data source using an .mdb file database, followed by use of SQL.  Although now deprecated in favor of the more current Access Database Engine formats, .mdb files are ubiquitous in the Microsoft world, one of the more popular file formats in which file databases are encountered.  


Example: Modify GPKG Geometry with SQL then Add Drawing - This topic provides a "Hello, World" example that shows a simple, but typical, task involving spatial data.  We will take a country-sized data set in GeoPackage (GPKG) format and change all areas in the data to the boundary lines for those areas and then save those boundary lines as a new table.  We add a spatial index to the table and create a new drawing to visualize the new table.


Example: Trace an Area in a Map over an Image Background - In a map with a drawing layer above an image layer (served dynamically by an imageserver), create an area object in the drawing by tracing over the outlines of something seen in the image layer below


Example: Style Applied to an Image Server Image - Because the Edit - Style dialog simply changes the way an image is displayed and not the data, it can operate on read-only data served by various web servers such as WMS REST servers.    In this example we look at every detail of creating a data source using a WMS REST  server and then manipulating the appearance of the display with Style.  We will connect to a WMS server that provides LiDAR data in various forms, including as terrain elevation.