How to build Stellarium landscapes from photographs of your own (2/4)

Creating the panorama with Hugin

Loading your pictures and initial fitting

Download your panorama snapshots from the camera to your computer. Create a separate subdirectory for the snapshots. If you use a photo management / archiving solution (e.g. digikam) for your camera, make sure that this also allows access to your pictures from outside this software (this is possible with digikam). All pictures must be in landscape mode. Otherwise, do not scale or pre-process your photos, especially not with regard to brightness, contrast, gamma factor or color!

Create another subdirectory, separate from the photos, for the meta, temporary and end files of Hugin.

Invoke Hugin and make the following settings under [File / Preferences]:

• Assistant: Switch off "Automatically align images after loading" and "Detect vertical lines", but switch on Celeste and "Remove outlying control points"
• Control Points Editor: click "Enable rotation search" and set start / stop angle to -30 / + 30 (compromise values ​​to limit CPU time, this has to be reworked manually in the nadir environment)
• otherwise keep standard settings.

Switch Hugin to Expert mode (set [Interface / Expert]) and set the settings for Geometric and Photometric to "Custom Parameters" in the lower area of ​​the main window under Optimize, so that the additional tabs "Optimize" and "Exposure" in the work area of ​​the main window appear. Lens type must be set to "Normal (rectilinear)" for a normal 35mm camera.

Now load the captured photos of your panorama into the application with the "Add Images" button or [Edit / Add Image] in the main window (another option is [Assistant / Load Images] in the Fast Panorama Preview). You can select all the photos you need at once. Save the project file now in the prepared folder for the meta files ([File / Save]) and get used to regularly backing up this project file with versioning each time you revise the panorama, so that you always have a reasonable fallback position, which you can go back to in case an overhaul turns out to be worsening.

If not already opened, invoke the Fast Panorama Preview (GL button in the main window). When invoked in this state of the project, all loaded photos appear stacked on top of each other in one place in the work area, as their position in relation to one another has not yet been determined by Hugin. In the projection tab, set the panorama type to "Equirectangular" and the field of view to the full area 360°x180°. With these settings Hugin will generate the panorama as a flat map that covers the entire sight sphere. This panorama type corresponds exactly to the "Spherical" landscape type in Stellarium.

Now start the primary process for the initial alignment (fitting) of the loaded photos by clicking on the Align button in the Assistant tab. All photos are examined in pairs for matches, which are then marked as pairs of control points, and based on this, a geometric and photometric optimization is performed. As part of this first fitting procedure - which can take around 1 hour on a modern dual-core system of the type described above, but can take 12 to 24 hours on older single-processor systems (and should not be interrupted under any circumstances!) - about 12,000 to up to more than 20,000 pairs of control points will be created for a panorama project comprising 100 to 150 pictures. After completion, these pairs of control points can be inspected for each pair of pictures in the main window under the "Control Points" tab.

With the successful completion of the primary fitting, a first preview of the entire panorama appears in the Fast Panorama Preview with an approximately correct positioning of the individual images, which also the appropriate values ​​for the positional parameters are now assigned to in the main window under the Optimize tab (or under the Photos tab if the "Display Positions" option is selected). The important position parameters are:

• yaw = azimuth of the image center relative to the anchor image (-180° to + 180°, + to the right)
• pitch = elevation angle of the image center (-90° (nadir) to + 90° (zenith))
• roll = angle of rotation of the image around the center of the image (-180° to + 180°, + clockwise)

In the initial fitting, an optimized angle value for the horizontal field of view (HFOV) of the camera is also determined, which can differ slightly from the value calculated from the lens focal length.

If the panorama horizon is uneven after the first fitting, the horizon should be roughly straightened (the exact alignment of the horizon will take place in further processing). Under the Move / Drag tab of the Fast Panorama Preview, in addition to the "Straighten" button for automatically straightening the horizon, there is also a manual option by manually setting the overall position parameters yaw / pitch / roll. Here you should orientate yourself on the displayed horizontal line that marks the mathematical horizon and only change one parameter at a time in 1° steps.

Eliminate the distortion caused by the handheld shooting technique

In the panorama preview created after the first fitting, if you look closely, you can see that the panorama often "swells" upwards by up to 10° above the mathematical horizon line (as an estimate: the mesh size of the displayed grid is 9°). The reason for this altitude excess is that during the nadir snapshots the camera lens is up to 20 cm closer to the ground than during the horizon snapshots due to the increasing tilt of head and camera with increasing inclination angle. This means that details on the photos taken at high angles of inclination are too large compared to the horizon snapshots. Calculating the field of view of the nadir snapshots back to the normal height of eye of the horizon snapshots results in a decrease of HFOVs for the nadir snapshots by several degrees.

In Hugin, this effect can be taken into account by assigning a new common lens to each group of snapshots that belong to similar angles of inclination, as indicated by the pitch parameter (highlight photos in the Photos or Optimizer tab of the main window, right-click to invoke the submenu and select [Lens / New Lens]). Typically 7 to 10 different lenses are obtained in this way, which correspond to the "recording lines" when shooting the panorama "line by line". These lenses can take on individual HFOV values ​​independently of one another.

Since the assignment of photos to new lenses results in an undesired coupling of photometric parameters, this coupling must be removed before work can continue. To do this, select the Exposure tab, click on Reset (Exposure / Color to EXIF ​​values) and then perform a photometric optimization run by clicking on the "Optimize Now!" button in order to regain the optimal brightness adjustment of the photos.

Now click on the Optimizer tab to initiate the first geometric optimization run after the new lenses have been created. Make sure that the position parameters yaw, pitch, roll (with the exception of the yaw parameter of the anchor photo) as well as the parameters HFOV and barrel (b) for all lenses are selected for optimization. Start the optimization run by clicking the "Optimize Now!" button and save the project file (versioning!) after completion. The most important change in this optimization run occurs in the HFOV parameters that now are being appropriately adjusted to the shooting technique and then will show a typical graduated decrease from horizon to nadir snapshots (e.g. in the case of a camera with a 54° field of view, the HFOV values typically stretch from about 54° for horizon snapshots to about 45° for nadir snapshots).

The altitude excess, i.e. the effect of the panorama "swelling out" above the mathematical horizon line, is thereby already significantly reduced, but generally not completely eliminated. In order to achieve a further clear improvement of the panorama with regard to these points, a geometric optimization run is now performed solely for the lens that includes the photos with the horizon shots (this is usually lens 0). To do this, first switch off all images in the Fast Panorama Preview and then individually switch on only those images that belong to this lens. Make sure that in the Optimizer tab of the main window the option "Only use control points between images selected in preview window" is activated, and start the geometrical optimization by clicking the "Optimize Now!" button. As optimization takes place only with respect to a certain group of images of the project, this run takes much less time than a full optimization. In many cases, limited to this group of images, Hugin can create a panorama with a very good fitting, only a few artifacts and in an almost correct horizon position.

If the horizon appears bent sinusoidally in the Fast Panorama Preview, the sighting direction of the panorama center is not aligned with the horizon and must be adjusted in the Move / Drag tab by dragging it by hand or by setting the position parameters (yaw / pitch / roll). Thereby positive (negative) values ​​cause

• at yaw a shift of the panorama to the left (right)
• with pitch an increase (decrease) of the panorama in the area of ​​the central point
• with roll a rotation around the central point clockwise (counterclockwise)

If the horizon is roughly straightened, it is now time to not only back up the project file as usual, but to output the current state of the panorama as a TIFF file for the first time by switching to the Stitch tab in the main window and pressing the Stitch button. The output panorama is later checked for correctness and corrected by calculating the azimuth and altitude angle of several landmarks.

Now prepare the further full optimization by switching on all images again in the Fast Panorama Preview and deactivating the optimization of the position parameters of all images of the lens that has just been optimized as well as its lens parameters in the Optimize tab of the main window. As a result, the position of these images is "frozen" in the subsequent optimization runs and the alignment of the remaining images is forced to these fixed images. Click on the Optimize button to start the next optimization run, which is generally associated with a further adjustment, in particular the HFOV parameters.

In most cases, further optimization runs are necessary (see the next but one section), since the output panorama generally contains artifacts that are triggered by parallaxes of nearby objects, by incorrect or insufficient number of image pair control points. The aim is a correctly positioned panorama that is as true to altitude angles as possible and free of artifacts.

Brightness, exposure and photometric optimization in Hugin

For the photometric fitting of the images, Hugin uses the provided EXIF ​​data (aperture, exposure time) to determine their respective exposure values ​​(EV) according to the EV-100 system, which then serve as initial values ​​for the photometric optimization process in which these values will be optimally harmonized with respect to overlapping images.

The exposure values ​​of the individual photos have an effect in Hugin on the panorama preview and creation in such a way that photos with higher EV values ​​(= lower exposure) are brightened accordingly, whereas those with lower EV values ​​(= higher exposure) are correspondingly darkened - namely relative to the average of the EV values ​​of all images involved in the panorama, which is also displayed as the initial EV value in the preview tab of the Fast Panorama Preview after each photometric optimization.

The preview in the Fast Panorama Preview already shows that in this way the panoramas taken in full daylight are generally clearly too bright. It is therefore advisable now to increase the EV value in the Preview tab of the Fast Panorama Preview by clicking the up arrow one or two times, which automatically darkens the preview panorama. Each click increases the EV value by 0.33, which would correspond to a reduction in exposure of around 21%.

Continued optimization: editing control points and setting masks

The main means for further optimization of the panorama is the addition and correction of image pair control points, for which a certain amount of time has to be set aside due to the large number of image pairs and control points (this isn't done "just in a moment"), possibly accompanied by masking unwanted image areas, in particular from your own body parts (feet / legs!) as well as third persons, moving objects or parallax errors.

Hugin has excellent capabilities for automatic control point recognition with CPfind, but such an automatism cannot do everything and also creates one or the other unsuitable control point or does not find any control points in an image pair, although the images clearly overlap. Therefore, an inspection of the image pair control points under the "Control Points" tab of the main window is essential.

It is essential to remove control points

• in the sky and on cloud formations (Celeste recognizes and removes most, but not all, of these control points)
• on water surfaces
• on objects that have changed their position between snapshots
• on objects with large parallax errors

Control points should be added manually

• for image pairs with poorly distributed control points
• in areas where the output panorama has artifacts
• for image pairs with few or no control points despite considerable overlap

In certain landscapes with special features, the setting of horizontal or vertical control points greatly facilitates the fitting and optimization process in Hugin. These types of control point pairs are not used to mark the same point in different photos, but rather to indicate that the points they relate are exactly horizontal or vertical to each other. These types of control points should never be set automatically, but always manually in the following situations:

• Vertical control points can be set if the images clearly show vertically oriented building edges with a length of at least 250 pixels. High masts (television towers, masts of wind turbines) of the appropriate length, where the horizontal center can be reliably measured, are also suitable for setting vertical control points. Trees, however "nearly" vertical, are completely unsuitable, as are Hundertwasser-style buildings.
• Horizontal control points should be set abundantly when areas of sea or of very large inland water bodies (e.g. Great Lakes in North America) form part of the horizon. However, the meteorological visibility must be great enough so that the sky / water boundary line can be recognized without any doubt. In these cases, horizontal control points offer a certain compensation for the fact that larger areas are not suitable for setting normal control points because of the water surface.

The problem that image pairs have no control points at all despite a large overlap mainly occurs in the nadir neighborhood and is due to the considerable rotation of the images relative to one another, which is typical with nadir shots. Manual intervention is required here, because insufficient cross-linking of individual images with neighboring images results in incorrect HFOV values ​​in the optimization runs.

The graphical representation of the image networking in the layout tab of the Fast Panorama Preview proves to be a great help in improving the cross-linking of the images. Thin gray lines connect image pairs that are not linked by common control points despite overlapping, and with a simple click on this thin gray line the respective image pair is brought into the "Control Points" tab of the main window in order to be able to edit it. For effective use of this tool in the nadir neighborhood, it is advisable to temporarily set the position parameter pitch to 90° in the Move / Drag tab of the Fast Panorama Preview in order to shift the nadir to the center of the panorama. In the layout tab, the images near and next to the nadir can then ideally be seen as concentric rings around the nadir and their existing or missing cross-linking can be easily recognized.

Images of image pairs are rotated when loading into the "Control Points" tab to match their position shown in the layout (in 90° increments), which makes it much easier to identify details that match each other in the image pairs and to set control points manually. At the end or interruption of this phase of work, set the pitch parameter in the Move / Drag tab to -90° in order to restore the normal position of the panorama. Then save the project file again.

How to eliminate persistent artifacts

In some cases, continuing to optimize the control points may not remove all of the artifacts in the panorama. Objects with larger parallax errors and their surroundings are usually affected by this. The following options are available for handling these artifacts:

• By setting include masks, certain photo sections can be forced to be included in the panorama. This can be used to have the "correct" representation of an object afflicted with parallax included in the panorama in artifact areas. The most suitable representation of an object afflicted with parallax is always to be found on the photo on which this object is closest to the center of the image when using the described recording method, since one can imagine a panorama composed of the smallest image elements that are projected in infinitesimal solid angle areas along rays arise from the imaginary observation point to an imagined infinitely wide sphere.
• Problematic individual photos, which in combination with the other photos cause artifacts in the panorama, can be switched off in the Fast Panorama Preview. As a result, they are no longer included in the creation of the panorama.
• Artifacts can also shift to another position if panoramas are produced in different ways from the same starting material, i.e. an artifact disappears in one part of the landscape, for which a new artifact appears in another part. In these cases there is often the option of using GIMP to intelligently tailor an overall panorama from such panoramas, from which the shifting artifacts are completely eliminated. The prerequisite is that the main part of the landscape on these panoramas retains exactly the same position in order to get seamless transitions. Use the "difference" feature of GIMP when superimposing the layers in question.
• These methods fail in the case of extensive objects with large parallax errors (e.g. sitting bench 2 m away). The only remedy here is to create a panorama without this object, by masking it out of all the shots involved and deleting all control points in the masked areas, and separately trying to create another panorama with control points only placed on the object in question. Then you load this second panorama as an equirectangular image into the first panorama and mask out the unwanted areas on it. Alternatively, you can try to leave both partial panoramas in Hugin completely separate and then merge them with GIMP.