|
Table of Contents
|
News and updates
NEW USERS GO HERE : http://www.rhysy.net/Code/Software/FRELLED/
This is the official FRELLED home page containing very quick guides to installation and getting started.
22nd November 2023 : Version 5.0 of FRELLED is now as complete and debugged as I can humanly make it, and the wiki is as complete as it's ever likely to be. No doubt there are still a few bugs lurking here and there that have escaped me, but they are now so obscure it's hard to think of any further tests to run. Any more bugs will have to be caught by accident, e.g. during unusual combinations of circumstances. I sincerely doubt there are any really serious problems remaining; everything should now be trivial.
FRELLED is in a state of feature freeze. New features are planned (see the Roadmap), but not until after publication.
If you're using FRELLED version 5+, please consider citing this paper. If you're using the older versions (please stop !) or want to cite technical details of how the volumetric display works, the older paper is more relevant.
Also please consider sharing this with whoever might be interested. After all this work, I'll be horribly disappointed not to have a user base.
Bug reports are always welcome ! To submit one, and/or opt in to the FRELLED mailing list, contact me at feedback @ rhysy.net. Updates are expected to come out a few times per year at most so this will add an absolute minimum of clutter to your inbox.
The one major change I'm considering is where this wiki and the code itself is hosted. Ideally I'd like to restructure this so that the different sections are all separate pages, e.g. installation, troubleshooting, reference etc. Especially the list of all the individual scripts (which is anyway still incomplete and not all that useful) doesn't really add anything except make the scroll bar intimidatingly small. But in all essential features, everything is done.
On to the next task : publication and promotion…
24th November 2023 : Naturally I found another bug : declinations south of -24 degrees would cause incorrectly-formatted axes labels and inconsistent use of the negative sign in the output printed in the terminal. If you downloaded the software in the last two days (or earlier), you should update it if you ever work with southern-sky data sets. Northern data should be completely unaffected.
Another seeming bug was that the velocities appeared to be wrong for certain spectral lines, but this turned out to be a data error, not a FRELLED error. The FITS keyword "RESTFREQ" or equivalent thereof needs to refer to the actual, physical rest frequency of the line, whereas in some cubes this apparently refers to the band centre frequency instead ! Well, there's nothing FRELLED or any other software can do in that case, so this is just something to be aware of. The "AGESCleaning.py" script in the "Extra scripts" subdirectory already allows modifying FITS header keywords.
11th January 2024 : A few more very trivial bugs have been squashed. FRELLED will now warn if the spectral axis does not have the correct keywords in the FITS header and direct you to the script to correct these. The Developer tab now contains a "Contour control" panel where you can set or disable a limit on how many vertices are allowed in contours and isosurfaces, which can help prevent extreme slowdowns and memory issues with very complex contours. Deleting a "WholeCube" region now longer messes up the main axes materials if the WholeCube itself had axes plotted; complaints about a shadeless material not existing when creating a WholeCube have also been fixed. None of these are in any way critical so don't feel the need to download the new version unless any of these are particularly important to you for whatever weird reason.
25th January 2024 : One critical but trivial bug has been fixed in another stealth update. Deleting regions would previously give an error message due to a single line with a missing indent. This is now fixed.
Additionally, I found a couple of other very niche issues which I haven't yet fixed : (1) the VR export scripts do not work well at all and need quite a bit of hacking to get them to do anything; (2) creating a camera when viewing a height map does not work correctly, but everything's fine if you set up the camera with height maps disabled and then enable height maps afterwards. Neither of these are very important, especially since VR use is currently minimal. I will wait and see if any other bugs emerge before tackling these.
I've also put in a link at the top to the new official FRELLED homepage, which is super short compared to this wiki and hopefully a lot less intimidating for new users !
2nd February 2024 : The VR export/import scripts have been replaced with simpler, faster, less destructive procedures. The import scripts now work both in Blender 2.91 and even Blender 4.0. Most likely they will work in all versions of Blender from 2.80 onwards but I haven't tested them. See the VR section for revised instructions.
22nd May 2024 : I discovered that the modern NED interface apparently no longer allows URLs to include filters, e.g. galaxy types and redshifts. This means that queries will only apply the filters selected using the classic interface. A new version of FRELLED as been uploaded which makes this explicit in the GUI and help files, but hopefully at some point NED will re-implement this, or possibly there's some other way to specify filters in the URL I'm not currently aware of. Downloading this is hardly necessary though, just use the "classic" NED interface option instead of the default "modern".
Second edit : Fixed a bug when loading files larger than the default maximum of 1500 pixels on any side. One of the scripts wasn't creating the textures correctly, causing a crash at the point of loading the images. Again, not strictly necessary to download the fixed version (though it may make life easier). If you do have larger files to load, the GUI options in the Developer tab to set up the file were not affected by this.
5th March 2025 : No more "stealth releases". With the latest version officially published, it's time for the first update. Version 5.0.1 is available here with full release notes here. The main changes are greatly improved performance of the graphical menus and fixing a bug that prevented any miriad operations from running on Linux, along with numerous other minor improvements and bugfixes. This page will only link to the latest version, but a full archive of all the post-publications releases is maintained on the main software page.
What is FRELLED and why should you care ?
It's a lovely program which is as much fun as a pile of kittens.
No but seriously, what is it ?
It's a set of Python scripts for Blender that turns this popular art software into something useful for astronomers. FRELLED stands for FITS Realtime Explorer of Low Latency in Every Dimension. Its name originates from an obscure Australian sci-fi show that absolutely nobody else ever watched. The point of FRELLED is to allow astronomers to view their data with the extremely powerful interface of Blender, which was explicitly designed for viewing and manipulating 3D data. The main features are :
- View FITS files with 3D volumetrics in real time
- Plot contours, isosurfaces, moment maps and more within the interface, including SDSS RGB maps
- Also allows visualising the data the old-fashioned way as a series of 2D slices optionally displacing them as height maps
- Enables creating presentation-quality figures with annotations
- Supports standard astronomical world coordinates though astropy
- Allows the user to create turntable-style rotation movies, animate the data display, and create time series animations
- Supports a variety of multi-volume rendering methods
- Performs basic spectral analysis via miriad, and allows viewing one cube while analysing another
- No coding required, everything is done in GUIs
- Runs on Windows and Linux (Mac is a work-in-progress)
FRELLED modifies the Blender interface to make it as astronomer-friendly as possible. Much of the Blender functionality is hidden so you can just get on with analysing data rather than having to learn everything about Blender, and many of Blender's built-in features are given dedicated buttons all in one convenient place for the same reason.
Sounds useful ? Read on ! Sounds pointless ? Bugger off then, away with you.
I don't like reading, I want this manual in video format instead
Not a problem. While I highly recommend reading the manual when things don't work as expected, videos can be a better way of getting started. These don't cover absolutely everything FRELLED does but they certainly cover the major features. Additionally, there's also an ALMA I-TRAIN session covering all the basics of using FRELLED in a hands-on, step-by step guide. The video itself can be accessed directly here and the accompanying FITS files can be downloaded here. Note that the I-TRAIN does not cover installation.
Installation and Getting Started
You can watch the playlist of 3 videos here (8 minutes total). Or you can see the individual videos below. However, you'll definitely find it easier to consult the wiki for this.
Installing Blender
Describes how to download a special version of Blender for FRELLED, how to set up shortcuts/aliases to run it, and the very small one-off command needed to configure Blender for optimal use. Only covers Windows, though it's very similar on Linux. 3 min 32 s.
Installing FRELLED
Covers the trivial process of downloading the FRELLED package and the marginally less-trivial best practises for keeping files organised. 2 min 21 s.
Interface
An overview of the Blender/FRELLED interface. Mostly should be intuitive, but if you've never used Blender before I highly recommend watching this. 2 min 19 s.
Displaying Data
You can watch the complete sequence of 6 videos here (26 minutes total), or the individual videos below.
Basic Data Display
Step-by-step guide to loading FITS files in the main volumetric display mode and how to improve the display without having to reload the data. Also covers the very basics of navigating in 3D space. 5 min 5 s.
Advanced Data Display
Dives a bit deeper into the display controls. Describes how to speed up loading data (FRELLED is, it must be said, not fast), how to have different transfer functions for colour and opacity, how to control the spectral axis scaling, when data can be reloaded without exporting it, saving display files, and other stuff. Not needed most of the time but definitely worth knowing about ! 5 min 41 s.
Velocity Maps
Tells you how to do the very simple process of using the spectral axis to provide colour instead of the data intensity. 1 min 14 s.
2D Mode
Prefer your data as a standard 2D image sequence ? This video takes you through how to use FRELLED's dedicated 2D mode and how to use its different navigation tools, as well as displaying the data as displacement (height) maps. 5 min 3 s.
Multi-Volume Mode, Method 1
Describes how to use one FITS file to provide opacity and another to provide colour. Mostly designed for simulations where you might want to display different components in the same space (e.g. temperature and density), but can also work with observational data. 2 min 18 s.
Multi-Volume Mode, Method 2
What if you have two completely independent data volumes occupying the same space ? No problem, FRELLED can load both at once. Though they must be of the same pixel dimensions and preferably have identical world coordinate systems. 7 min 7 s.
Data Analysis
You can watch the complete sequence of 6 videos here (29 minutes total), or the individual videos below.
World Coordinates
Describes how to position the cursor and the viewer at a specified world coordinate position, e.g. RA and Dec, also how to change the spectral axis, scale labels, show slices though the volumetric data, and other essential ways of navigating through FITS files. 4 min 51 s.
Cataloguing Data
Covers the process of cataloguing sources by generating mask regions : how to place and scale them, change their display options and naming scheme, and how to export regions and import them from external catalogues. 6 min 52 s.
Running External Queries
While you can do a lot within FRELLED, sometimes you need to be able to access external information. FRELLED lets you directly query the SDSS and NED either using regions or just from the cursor. This video describes how to get the most out of those features. 2 min 36 s.
Mapping
Once your regions are generated, you can use them to plot various kinds of contours and other maps. This video doesn't demonstrate all of them but does show the major features : how to set data ranges and adjust contour thickness, how to toggle the display of the different maps etc. 6 min 49 s.
Isosurfaces
While these are basically the same as the other maps, they have enough unique requirements and features that they get their own video. 3 min 54 s.
Spectral Analysis
Describes how to use FRELLED's interface to miriad for spectral line analysis with mbspect. 3 min 44 s.
Figures and Animations
You can watch the full 20 minutes of 4 videos here, or see the individual videos below.
Annotations
Before creating a saved image, why not add some labels to it ? This video describes how to do that. 5 min 5 s.
Figures
Saving a figure in FRELLED isn't always quite as simple as pressing a button (though it often is). This video walks you through the process and also looks at how to keep the display organised when you've got dozens of annotated regions set up in a single file. 4 min 21 s.
Rotation Animations
There are two main ways of making animations in FRELLED. This first video covers the easy method, of making turntable-style rotation animations. 4 min 44 s.
Data Animations
The second, more powerful method lets you animate the data itself, for example animating a sequence of FITS files as a time series or even changing the colour transfer function. This can produce some extremely interesting effects. It isn't difficult to do, but everything must be done very precisely and it can be slow. 5 min 45 s.
Getting FRELLED up and running
Installing and running FRELLED isn't quite as simple as downloading a single file, but the procedure has been designed to entail a minimum of bloodshed with three mostly-simple steps :
- Installing Blender and adding several modules to its built-in Python distribution
- Running a small one-off setup procedure to configure Blender
- Downloading the FRELLED package and opening the FRELLED.blend file using Blender
A first-time installation requires all three steps, but upgrading to new versions of FRELLED can usually be done using just step 3.
By far and away, installing the extra Python modules can be the most problematic. There are two ways to do this : either (a) download a version of Blender with all the modules pre-installed, plus the FRELLED package itself (the easy method) or (b) run a script to install all the modules from scratch (the hard method). In either case, Blender's Python installation is kept completely separate from other versions and should run happily alongside any other versions the user has installed.
Currently, the easy method is available and tested only on Windows and Linux. Only the hard method is possible for now on Mac, and this is not fully tested.
Installation : the easy method
1) Installing and configuring Blender
Download Blender either for Windows or Linux (Mac version coming soon) using these links :
This special version comes with all the necessary Python modules pre-installed. Unpack the files wherever you want to keep Blender (your usual "Documents" directory or equivalent therefore will be fine). On Linux, you may need to make the Blender binary file (simply named "blender") executable, which can be done in the usual way, i.e. "chmod +x blender". A small, one-off step is then needed to configure Blender to run FRELLED :
1) Navigate to the directory where you installed Blender
2) a) Linux users : in a terminal, run the command :
./blender -y --enable-autoexec Setup.blend2) b) Windows users : shift+right-click in the file browser and choose "Open PowerShell here", then run the command :
./blender.exe -y --enable-autoexec Setup.blend2) c) Mac users : in a terminal, run the command :
./blender.app/Contents/MacOS/blender -y --enable-autoexec Setup.blendAll operating systems please note : The command above will run a setup file to make a couple of essential changes to Blender's preferences (enabling multi-sampling and automatically running registered Python scripts). Blender will then terminate itself automatically, so you will likely see Blender appear briefly on the screen and then vanish. Note that you may also encounter some non-critical warning messages during this process but these can be safely ignored (see the "Checking FRELLED installed correctly" section for these; if you see something other than these you may have problems.
2) Running Blender
A) Linux
The easiest way is to create an alias, which you can do with something like the following :
alias blender='/BLENDER/blender'Where /BLENDER/ is the full path to the directory where the Blender binary file is located. After running this command, typing 'blender' will start the correct version of Blender wherever you are.
To make this permanent, add the same command to your .bashrc file. For a more general introduction to setting up aliases, see this guide.
Linux - optional extras
You can also create a graphical shortcut (symbolic link) to Blender with the following command :
ln -s /BLENDER/blender ~/Desktop/blender.lnkWhich would put the shortcut on your desktop. Note that if you run Blender from the GUI, you'll need to use the "Window -> Toggle system console" button inside Blender to ensure that Blender has somewhere to report progress and any errors encountered. This is not strictly necessary, but it can make life a lot easier.
Finally, you can also run Blender's version of Python directly (without running Blender itself). The Python binary file is located in /BLENDER/2.79/python/bin. From that location, simply run :
./python3.5mThis gives you access to Blender's Python will all its useful extra modules. Of course, you can also set up an alias to run this if you need to use it frequently.
B) Windows
Create a shortcut to Blender by the following :
- Right-click on the blender.exe file and choose "Create Shortcut"
- Right-click on the shortcut and choose "Properties"
- Replace the entry in the "Target" field with the following :
"C:\BLENDER\blender.exe" --start-console"Where "C:\BLENDER\" is the full path to the directory containing the blender.exe file. And that's it. You can move and rename the shortcut however you like - placing it on the desktop would probably be a good idea. Start Blender by double-clicking the shortcut just like any other program. Note the "—start-console" switch ensures Blender will open two windows : the main interface, and a console where FRELLED will report progress status and any errors encountered.
(It's apparently possible to set up aliases in Window's PowerShell environment too, similar to those of Linux, but I've never tried this.)
C) Mac
Please note that as I do not have a Mac, these procedures are far less rigorously tested than on the other operating systems !
Running Blender from the terminal and creating an alias should be much the same as on Linux. The only difference is the command to start Blender, which is :
./blender.app/Contents/MacOS/blenderTherefore the alias command will be :
alias blender='/BLENDER/blender.app/Contents/MacOS/blender'Again, where "/BLENDER/" is the full path to the directory where Blender is installed. The procedure to make the alias permanent by adding it to your .bashrc file is identical to Linux.
On Mac, Blender's Python can be accessed in /BLENDER/blender.app/Contents/Resources/2.79/python/bin. From there, run "./python3.5m", exactly the same as for Linux apart from the location of the file.
3) Installing and running FRELLED
Download the FRELLED zip package from this link (same for all operating systems) and unpack it somewhere. This will be your master copy. For each FITS file you want to view, it's recommended to copy this entire directory, placing the FITS file you're interested in inside it.
The only requirements for where to place FRELLED are that it should be somewhere you have write permissions, and separate from the directory where you installed Blender itself.
To run FRELLED, start Blender as per step 2. Then inside Blender, at the top left, select File -> Open and then use Blender's file browser to choose the FRELLED.blend file in the FRELLED directory you're working with.
Installation : the hard method
The easy method is much preferred because you don't have to worry about installing the Python modules, which can be a right pain. But if you really can't get it to work (see the Troublehsooting section below), you can try and do things the old-fashioned way.
Installing and configuring Blender
Download the version of Blender (2.79b) appropriate for your operating system from this link (not the same as in the "Easy Method" above). Don't use any other link. No, really, this is important : DO NOT USE THE LATEST VERSION OF BLENDER, USE THE LINK PROVIDED HERE ! FRELLED is designed for this older version of Blender (2.79) and will not run on later versions. I'm emphasising this point as strongly as I possibly can because this is by far the most common "error" reported.
The second very important point to be aware of is : you must download the compressed version (.zip or .tar.bz2), NOT the executable installer files (.msi, .dmg). This will ensure your Blender installation is completely self-contained and kept safely apart from any other Blender and/or Python installations on your system. Mixing different installations together is a bit like having a safari park in which the lions and the zebras are all kept in the same enclosure : very exciting, but not in a good way.
Since getting the right version of Blender is crucial, the most typical versions to download are :
Unzip the Blender package wherever you want to keep Blender installed.
Installing FRELLED
Next, download the FRELLED installation packages file (same for all operating systems) into the directory where you placed Blender and unpack it there. This should add three files :
- MakeFRELLEDScript.py
- font_manager.py
- Setup.blend
These must all be in the same directory as the Blender .exe or binary file.
Next, navigate to this directory (in a terminal on Linux or Mac, or PowerShell window on Windows) and run the following :
Linux :
./blender --background --python MakeFRELLEDScript.pyWindows :
./blender.exe --background --python MakeFRELLEDScript.pyMac:
./blender.app/Contents/MacOS/blender --background --python MakeFRELLEDScript.pyThis will create a file "InstallPythonModules.txt". This contains detailed instruction on how to proceed including the commands you need to run. Since the instructions themselves are comments, if you want you can just copy and paste the whole thing into a terminal/PowerShell and let it run. This isn't recommended, though, because it will take some time (~15 minutes) to run, and running each command one at a time will let you spot any problems much more easily.
Note that this script accounts for the operating system and generates different commands depending on whether you're on Windows, Linux, or Mac. It also describes how to configure and access Blender and FRELLED, much as in the "easy method" described above. Basically you can follow the easy method from this point onwards, except that you'll need to use the version of Blender you just set up rather than the file given in that section.
Note that the Mac installation will currently only generate the commands to install the Python modules and omits the command to configure Blender afterwards. This is minor, but see the first part of of the "easy method" section for the command. Run this once all the Python modules have been successfully installed.
Upgrading FRELLED
While you can install new versions of FRELLED in exactly the same way as the first-time installation method described above, you don't usually need to do all this. If you've got Blender working with all the correct extra Python modules, you can just download the new FRELLED files and open them with Blender in the same way as before.
Having different versions of FRELLED running alongside each other is easy as they've just separate .blend files. When you download a new version, keep a copy of its master files somewhere so you can distinguish them from the previous version. As before, for each FITS file you want to analyse, create a directory and copy both the FRELLED and FITS files there. You can then open the new FRELLED.blend file using the same version of Blender.
A major downside of FRELLED's architecture is that this means that once you've done all your analysis on a cube and save the resulting FRELLED.blend file, you cannot easily load this into a new FRELLED version. This is something that may be addressed in the next major upgrade. You can at least export regions you create in an old version and import them into new versions (see the "Region setup" panel), but anything more than this is currently complicated.
During major version changes of FRELLED, and occasionally during smaller releases, there may be changes to the Python modules FRELLED requires. When this happens you will have to download the new precompiled version of Blender as well and follow the full installation process. You can either keep your old version of Blender or replace it with the newer version - the latter is probably better as new versions should be backwards-compatible with the old, except in rare cases. Upgrading Blender should be necessary only rarely. When it does happen, this will be clearly indicated, and an archive will be maintained of all old precompiled Blender versions as well as the FRELLED files. But this is a problem for the future.
Installation troubleshooting
Problems encountered during the installation
Problems finding pip (with "hard method" of installation)
Ordinarily the "ensurepip" command will create several pip executable or binary files (e.g. pip3.exe and pip3.5.exe) in the /BLENDER/2.79/python/Scripts (Windows) or /BLENDER/2.79/python/bin (Linux) directory. However, on Windows (Linux and Mac are untested), this may not work if the user already has a pip installation. The installation script ("InstallPythonModules.txt") tells the user which directory to check if this happens, something like :
C:\Users\MyUserName\AppData\Roaming\Python\Python35If the pip executable files are there, the script provides the command to copy them to the correct location. DO NOT try and omit the "./" prefixes in the script to run the user-installation of pip - it is essential that pip is located in the correct directory. If it isn't, pip can report no errors during the module installation but they will crash horribly when used !
If the pip files aren't found even in this this AppData directory, try doing a file search and copying them over manually. I don't know if this will work, however.
If all else fails, then, at your own risk, try deleting the above ../AppData/…/Python35/ directory and re-running the "ensurepip" command in the InstallPythonModules.txt file. That should force the creation of the pip executables in the correct location and you can proceed with the installation.
Most problems with installing or using individual modules relate to pip. If you encounter any of these, including those you might only discover when trying to use FRELLED in anger, try the above steps to be sure you're using the correct version of pip from the correct location to run the installation.
While sometimes pip can run without errors but still install things incorrectly, the "spectral-cube" module is known to be particularly prone to problems. Errors such as "no matching distribution" or "Command errored out" are most likely the result of a problem with pip. The same fixes as above should correct this.
Problems after installation
Checking FRELLED installed correctly
Sometimes errors are only apparent when you try to load the FRELLED.blend file. If FRELLED appears to load correctly but just doesn't seem to be working, here are some basic checks to do :
- Check the console (or the terminal where you launched Blender). Sometimes error messages crop up here when they were non-fatal; Blender's approach is to keep going in spite of problems unless it really can't continue. Windows users take note : Blender should open with two windows, one for Blender itself and the other for its message-reporting console. If you don't see this, select "Toggle system console" from Blender's "Window" menu in the top banner. Linux users : always launch Blender from a terminal to be sure you can see the error messages.
- Check the FRELLED GUI is correct. On the left hand side of the screen you should see a series of tabs : FITS Display (open by default), FITS Analysis, FITS Figures, and FITS Developer. If they are not all present, or not in that order, something has gone wrong - check the console for errors. See the videos above to see what FRELLED is supposed to look like.
- Try to do some basic checks on the functionality : using the example M33.fits file provided, try doing "Get levels"; "Load data"; check the coordinates are found correctly in the Navigation tab; try generating renzograms and isosurfaces. The last are particularly useful checks as the scikit-image (skimage) module is known to be problematic for installation.
If these diagnostics encounter any problems, or FRELLED has more serious problems (like crashing immediately when trying to load the .blend file), see the fixes described below. Most likely, something has gone wrong with the installation of the Python modules, even if no errors were reported during the installation phase.
There are also a couple of very common non-critical warning messages that can be safely ignored. You may see something like :
AL lib: (EE) UpdateDeviceParams: Failed to set 44100hz, got 48000hz insteadAnd/or :
Error: Not freed memory blocks: 1, total unfreed memory 0.000076 MBThe latter only occurs on exiting Blender. Neither are really errors but are only warnings. If it annoys you, there's a fix for the first one described here. I don't know of a fix for the second but since it only occurs when Blender quits, this can be safely ignored. I get this myself and have never bothered with it.
If you're finding that Blender quits when trying to load FRELLED, don't be distracted by these warning messages. Instead the problem likely lies in one or more of the possibilities described below.
Checking Python modules installed correctly
While the installation procedure by both methods has been set to use module versions which are known to work, it is still possible that some bugs or conflicts remain - or more likely (especially by the "hard method") that one or more failed to install correctly. You can diagnose this by trying to import the modules in Blender's Python installation manually.
There are two ways to access Blender's Python :
- Inside Blender. Use the drop-down Window Selector (see Interface video at 1:36) to select the "Scripting" window. You'll see a Python interpreter is running at the bottom.
- Outside Blender. Navigate in a terminal to /BLENDER/2.79/python/bin and run the command "./python.exe" (Windows) or "./python3.5m" (Linux or Mac).
Wherever you access Blender's Python, you can type the following commands to check that all the necessary modules installed correctly :
import numpy
import matplotlib
import astropy
import scipy
import skimage
import PIL
import spectral_cube
import parmapIdeally when you run this nothing will happen. If there's a problem you might get an error message, and sometimes if you're doing this from inside Blender itself, Blender will crash to desktop. Be sure to run each import one at a time so you can pinpoint the source of the error.
Although the module versions are explicitly specified when running from the "hard method" installation script, you may want to check these are correct :
numpy.__version__ == 1.18.15
matplotlib.__version__ == 3.0.3
astropy.__version__ == 3.2.3
scipy.__version__ == 1.4.1
skimage.__version__ == 0.15.0
PIL.__version__ == 7.2.0
spectral_cube.__version__ == 0.4.5
parmap.__version__ == 1.5.2All of these should return "True", except for parmap which doesn't have the usual " __ version __ " attribute. Check this manually by opening /BLENDER/python/lib/site-packages and looking for the parmap "dist-info" folder, which is named according to the version.
Note that the module versions have been carefully chosen to ensure that they all play nicely together, as some of them call each other. In particular, check the versions of spectral_cube and parmap. They MUST be the versions specified here as the later versions do not work. This is partly a result of the modules being recoded and partly because Blender 2.79 is no longer maintained, so has some incompatibilities with later module versions.
If you have the wrong version of a module installed, see "Problems finding pip" above - if you've run pip from the wrong location, it will sometimes be unable to find the correct version of certain modules (especially scikit-image a.k.a. skimage) or even finding the modules at all. See also the "Known Problems" listed below.
Known problem : errors with "__ init __.py"
Since the modules call each other, when one has a problem, usually the rest have a problem too. This means that the situation can appear worse than it really is, with lots of modules reporting problems but actually only one is really responsible. For example, a known problem encountered as a result of improper installation of some modules (especially matplotlib) is an error message referring to " __ init __ .py". If you see an error like this you can try deleting the offending file, but most likely you'll need to reinstall the broken module. Again, this is very prone to happening if you use the wrong version of pip or launch pip from the wrong location.
Known problem : crashing when trying to import matplotib and/or scikit-image (skimage)
One of the most frustrating bugs is a hard crash when Blender tries to import matplotlib, either directly or via skimage. This can cause a failure when loading FRELLED, typically with one of the FITS tabs in the GUI missing. When trying this manually from a terminal, Python itself crashes with no errors. This was an extremely difficult bug to track down : it happens when either matplotlib itself or another module (usually scikit-image) tries to call the standard "import matplotlib.pyplot as plt" or something similar.
What's happening is that there's a matplotlib file called "font_manager.py" which is trying to import a file "fontlist-v300.json" which does not in fact exist. The installation script (for the "hard method") already includes a fix for this : it copies a modified version of "font_manager.py" into the correct directory, replacing the file matplotlib itself provides. This should work on all operating systems but has not been robustly tested. There are several alternative options for manual fixes, though I don't know how well they work :
- In the /FRELLED/ExtraScripts/ directory, both the files "fontlist-v300.json" and "fontlist-v310.json" are included. On Windows you'll have a hidden directory "C:\Users\Me\.matplotlib", or on Linux "~/.cache/matplotlib" (Mac location currently unknown). Copy both of the .json files here.
- Do a file search to find files "fontlist-v*.json", which is likely to relate only to matplotlib (Windows/Linux users, look in the directories above). Note the version number. In "/BLENDER/2.79/python/lib/site-packages/matplotlib/" (Windows) or "/BLENDER/2.79/python/lib/python3.5/site-packages/matplotlib/"(Linux), edit the file "font_manager.py". Look for, " __ version __ " (should be on line 951). Change this to the version number of the json file you found. This is essentially the fix the installation script itself does.
- Install a different version of matplotlib : it's possible earlier versions might not have this bug, though I'm not about to go through every version available. 3.0.3 seems to be the latest that Blender can handle so there's probably no point trying later versions.
To try the last option of installing a different version of matplotlib, go to the directory where you have your pip binary or executable (as above, this must be in /BLENDER/../Scripts) and try :
./pip3 install --target="\BLENDER\2.79\python\lib\site-packages" matplotlib==3.0.3 --upgradeReplacing "\BLENDER" with the full path to your Blender installation and 3.0.3 with something else. I have no idea which, if any, might work.
Known problem : crashing when trying to import matplotib and/or astropy
Another crash-to-desktop (without error messages) can occur because of another problem relating to matplotlib or more commonly astropy. If the "font_manager" fix described above doesn't work, it's possible you're missing a couple of other packages relating to matplotlib and/or astropy. Normally these are created automatically when the modules are first imported; if you don't have them (e.g. if you installed FRELLED somewhere you don't have full write permissions), you can download them as below.
1) Download the extra matplotlib file here. Unzip this depending on your operating system.
Windows :
C:\Users\YourUsername\.matplotlibLinux :
~/.cache/matplotlibNote that this directory is hidden. It should now contain the empty "tex.cache" directory and the "fontlist-v310.json" file.
2) Download the extra astropy file here. Unzip this depending on your operating system.
Windows :
C:\Users\YourUsername\.astropyLinux:
~/.astropyAgain this is a hidden directory. It should now contain a "config" directory containing two configuration files.
Now try loading FRELLED again.
Permission errors
On Linux, you may get as far as generating the axes as the FITS file and then encounter an error relating to permissions when running the external script. In this case, what's likely happening is that Blender can run its Python internally but the external Python binary is not executable. The error message ends with something like :
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission deniedTo fix this, navigate to the Python binary directory e.g. /BLENDER/python/bin and run "chmod +x python3.5m" to make the Python binary executable. You can also test if this is problem by running the Python binary directly e.g. "./python3.5m". If it runs, FRELLED should now work correctly. If not, something else is going on.
Blank display
If you try and load data and you see nothing at all (i.e. the default black background), or just a set of axes, there are a couple of possible issues - assuming that you've set the data range values to something sensible and are confident that you should see something. If you see the axes, the first thing to check is in the Analysis tab : ensure that "Show FITS file" is enabled. If that doesn't work, or you don't see the axes, check in the Developer tab that "Modal ops" is enabled. It's easy to accidentally disable this via the escape key, and it could well be that somewhere in FRELLED it's been disabled and not re-enabled correctly.
If you encounter this bug and can reproduce it, please contact me, especially if the above steps don't work !
Performance drop
Sometimes when you reload a file it may be unbearably slow. This happens because previously the installation instructions recommended starting Blender using the switch "—enable-new-depsgraph". I no longer recommend using this and in fact strongly suggest removing it. The issue is Blender's dependency graph, which it uses to update the scenes correctly. The old version of this, used by default, causes annoying and pointless "DAG zero" messages in the terminal, which users understandably found distracting and confusing. The switch told Blender to use a newer version, which doesn't produce these messages but has the (much worse) effect of this severe performance penalty.
This error appears to happen only if you save and re-open a file in 2D mode. In 3D mode things should be okay (though, strangely, 3D mode does run slightly better with the switch enabled - this is under investigation). The problem can be fixed simply by removing the "—enable-new-depsgraph" switch from the shortcut or alias you use to start Blender. If it happens in other situations, please contact me !
Uninstalling FRELLED
Obviously, the only reason to uninstall FRELLED is because you think something got broken during the installation process and you want to start with a nice clean setup. Since Blender is installed from unpacking a .zip (or similar) file, you can simply delete the directory where you installed it. However, a couple of the other Python modules installed for FRELLED also create directories elsewhere. These are :
Windows :
C:\Users\YourUsername\.matplotlib
C:\Users\YourUsername\.astropy
C:\Users\YourUsername\AppData\Roaming\Python\Python35Linux :
~/.cache/matplotlib
~/.astropyDelete these directories as well to ensure FRELLED and every last ghastly trace of it is removed from your system.
Using FRELLED
Quick start guide for beginners
The story of Princess Olivia and the magical moose whose name no-one could still quite remember… part three ! To be written eventually. For now you'll have to make do with the video guides.
Limitations : what to know before you begin
FRELLED is designed for astronomy FITS files, which in general consist of a header containing essential keywords about the file and at least one 3D array of data values. In principle any data set that properly conforms to the FITS standards can be loaded into FRELLED, but it's designed primarily for radio astronomy data. This means that other data sets may not work. If you encounter a file that stubbornly refuses to load, please get in touch ! Email feedback @ rhysy . net
The FITS format doesn't impose many strict constraints on the data, but for the sake of sanity, FRELLED is a bit more restricted in terms of the data it can load. In particular :
- The FITS file must consist of a single 3D array. Data sets containing multiple arrays cannot be loaded at present. A script is provided to reshape 4D arrays (often used in radio astronomy) into 3D versions if necessary, which FRELLED will warn you about if it encounters a problem.
- Data sets should have no more than 1500 pixels on a side, due to limitations of Blender. This can be circumvented to some extent if it's really necessary, but by far the easiest solution is to truncate your file to something smaller.
- Partly due to the Python modules used by FRELLED, certain FITS header keywords must conform to particular standards that are not enforced by the FITS format itself. For example the spectral axis units must be exactly "Hz", not "HZ" or "MHz" or any other variation. Some error trapping is done for this within FRELLED, which will warn you to use the supplied "AGESCleaning.py" script to make the appropriate modifications if necessary. However it's impossible to guarantee that every possible error has been accounted for ! Do get in touch if FRELLED doesn't support a file because of some formatting issue.
- FRELLED displays all data in Cartesian projection, including all-sky data.
- Very small cubes (< 2" across) may have problems as the code generating the axes is not designed for such myopic detail. This will be addressed in a future release.
- Likewise, FRELLED is designed for spectral axes spanning >~10 km/s. Very high spatial resolution cubes may have problems. However, as with spatial resolution, FRELLED will revert to simple pixel coordinates if it can't handle the world coordinates correctly - the axes will be labelled in pixel coordinates but the data itself should display correctly.
Reference guide - the basics
This section describes the major operational features of FRELLED : the screen layout, how to navigate in a 3D file, and how to keep FRELLED nicely organised for larger projects.
How to interact with Blender
FRELLED has been designed to make the user interface of Blender as astronomer-friendly as possible. To that end virtually all operations can be accessed via GUI buttons. However, there are still a few basics to be aware of :
- Keyboard buttons depend on where the mouse is. When you do use a keyboard-based shortcut, you need to have the mouse over the correct section of the FRELLED screen. For example, trying to rotate the view with the "6" key on the numeric keypad won't do a darn thing unless the mouse is in the 3D viewport
- Numerical sliders can have precision entries. Sliders have small arrows at either side. Left click and drag to change the value using the mouse, or just left click and type an entry to set a value precisely. Some sliders have "soft" minimum and maximum values. These prevent the user from accidentally dragging the values to something generally insane, but you can still type in a value outside this range. Others have hard limits that cannot be circumvented.
- Operations can be cancelled with the right mouse button. This includes editing slider values and textbox entries, as well as when manipulating objects. They can also be cancelled with the escape key but do NOT do that as this interferes with other FRELLED operations. If you accidentally do this, see the modal operator button in the Developer tab.
- In the 3D viewport, left click places the cursor, right click selects the nearest object. The 3D cursor is just a handy marker and does nothing to the file itself. Selecting objects is useful for various operations I'll cover later. Also try to avoid the Tab key, which lets you edit the constituent vertices that define Bender objects : if you do this, press Tab again to go back to object mode.
The FRELLED interface
The major features of the FRELLED layout are shown in the screenshot below. They are as follows :
- 3D viewport : the main display area, where the FITS data and various annotations will be displayed
- 3D cursor : shows where any new regions and annotations will be created. Also useful for navigation
- Screen selector : choose between different layouts of Blender's tools. Almost always only the default "FRELLED" is needed. "Scripting" and "Standard Blender" can be useful for debugging while "Animation" is rarely needed
- Scene selector : avoid using this ! Blender has different workspaces called scenes, but the choice of which one to show is done within the FRELLED GUI and doing this manually can mess everything up
- Timeline controls : allowing control for animations, e.g. which frame is currently shown, range of frames to animate, etc.
- FRELLED tabs : controls which FRELLED menu is currently shown in the left-hand GUI
While control of FRELLED has been designed to be via GUI, it's also possible to access all of its operations via the keyboard. With the mouse in the 3D viewport, press the spacebar to bring up Blender's all-purpose menu and start typing the name of an operation (clues to what these are called can be found in the tooltips for each button). For example typing "add region" will bring up the operator to add a new region, used for marking parts of the data, enabling you to do this without having to use the GUI menu on the left at all.
Navigating in FRELLED
Navigation in a 3D environment is necessarily a bit more complicated than with a 2D image. While in principle you could do almost everything with the dedicated "Navigation" panel, to freely explore a data set it's well worth learning the keyboard and mouse shortcuts. Note you can normally click the scroll wheel to use it as a middle button.
Mouse controls
- Scroll wheel : zoom in and out
- Shift+middle button drag : pan
- Middle button drag : rotate
- Shift+scroll wheel : pan vertically
- Ctrl+scroll wheel : pan horizontally
- Ctrl+middle button drag : smoothly zoom in and out
- Left click : place the 3D cursor at the mouse position on the screen
Keyboard shortcuts
On the numerical keypad :
- 5 toggle between perspective and orthographic view
- 1 face-on "XY" projection, usually the sky view (RA-Dec)
- 7 top-down "XZ" projection, usually RA-velocity (or spectral axis)
- 3 side view "ZY" projection, usually Velocity-Dec
- Ctrl+[1,7,3] : same projections as above but from the opposite direction
- 2, 4, 6, 8 rotate the view in 15 degree increments
- ALT+Home centres the view on the 3D cursor
- Del (or period) key centres the view on the mean position of all selected objects
Keeping FRELLED organised
When you just want to view a FITS file, FRELLED is not much different to any other FITS viewer. However, when doing a more extensive analysis - e.g. cataloguing sources in a data cube, there are few points that will make life much easier.
Save your work. Whenever you significantly modify a file, e.g. by adding regions, use Blender's File menu to save the FRELLED.blend file. This will preserve the entire FRELLED setup including the display of the FITS file, the regions created and their visibility status. You may want to rename the file to something distinctive so you remember what it is in future, but this is optional. Reload this file from Blender's file menu - this will also avoid having to reload the data into FRELLED, which can be slow for large files.
IMPORTANT : A Blender bug means that if you save and re-load a file in 2D mode there can be an extreme slowdown in the GUI display. This can be overcome by switching to 3D mode and then back to 2D.
Keep one FITS file per FRELLED folder. Although the FRELLED file browser lets you choose FITS files located anywhere you can access, it's not very happy about actually loading them unless they're located in the same folder as the FRELLED.blend file. Admittedly this is a bit of a mixed blessing - it means you can end up with more files floating about than you might like, but it also means each folder is a self-contained, easily-duplicated project.
It is possible to use multiple FITS files in the same folder, however, there are some caveats. For multi-volume rendering having multiple files in the same folder is actually essential. But if you're analysing multiple cubes, while there should be no problem in visualising them, some of the analysis tasks create external output files (e.g. maps and figures). If you use the same names for the regions created, these can be overwritten !
Choose how to organise your panels. Each panel in the FRELLED GUI has a little triangle symbol at the top-left which is used to expand and collapse it. This means you can customise your display to see what's most relevant to you. Also, at the top-right of each panel is a rectangular series of dots. Click and drag this to move the panels around and re-order them.
Using 2D mode
While FRELLED's main selling point is that it can examine FITS files in their full 3D volumetric glory, it can also view them more conventionally as a sequence of 2D images. In 2D mode, use the timeline controls at the bottom of the screen to advance through the data set in each projection. There are several ways to do this. Next to the "start" and "end" slider boxes, another box sets the current "frame", i.e. the current slice in the projection currently shown. Above this, the timeline itself shows the available frame range in light grey, bordered by darker areas outside the available range. The green bar shows the current frame. Left-click (and optionally drag) within the light grey area to change the current frame. Lastly, you can also advance and decrease the current frame using the left and right keyboard cursor keys : hold down shift as well to move by ten frames at a time.
While in 3D mode rotating the view to arbitrary angles is a feature, in 2D mode this can become something of an annoyance. This makes it particularly important to remember the numeric keypad shortcut buttons : 1, 3 and 7, which switch between the major direct viewing angles. Note you can also access these with additional special functionality in 2D mode via the Navigation panel in the Analysis tab.
Reference guide - full panel descriptions
You can get help at any time for each panel by clicking on its "?" icon. This will bring up a text file containing a brief description of what it does, which is reproduced and expanded on here.
FITS Display tab
This tab contains all the tools needed to control the display of the FITS data loaded into FRELLED. Control of the display of the axes and other annotations is done in the other tabs.
Main display settings
This panel sets the most essential parameters for displaying FITS data.
The FITS file chosen here is the primary FITS file to display. It is possible to overlay a second file in various different ways (see Multi-volume display settings), but by default only a single file need be chosen. Choose the file using the little folder icon, which will open a file browser within Blender. When you have chosen a file, you can also left-click in the box to the left to specify a file name by hand, but this must be in the same directory as the original file.
The 2D/3D toggle selects the viewing mode. In 2D, the file is shown as a series of slices, with the current slice being controlled by the timeline slider at the bottom of the screen. In 3D mode the file is shown as a 3D volume.
The "log scale" checkbox sets whether to use a logarithmic scale for the colour transfer function. If the data range selected includes negative values, the data is translated linearly to make it positive before computing the logarithmic values. The translation increases all the data values by the minimum value of the data to display plus a very small value, 1E-10. If you're working with extraordinarily small data values, you might want to consider rescaling the data before viewing it in FRELLED.
"Get data levels" selects the data range to display based on the adjacent "lower" and "upper" percentage sliders (the actual data values are shown - and can be set independently - in the two sliders immediately below these). For example, by default this will set the lowest data range shown to be at the 50th percentile of the data - below this, all the data will be black and completely transparent. The upper data range, above which all the data will be saturated white and at the maximum opacity level (see "Loading and display controls" panel) will correspond to the 95th percentile of the data.
Pressing this button will also compute the standard variation of the data and print some useful values (1, 2, 3, 4 sigma) to the terminal. This can be a better way to suggest the range of data to display, but in the end there is no foolproof way to select a good range - it is strongly dependent on the data distribution, and may need some experimentation to find a good range.
If the data range has a higher minimum than maximum value, the colour transfer function (see below) will be reversed, e.g. the default is that faint emission should be transparent black and the bright emission opaque white, which would become faint emission shown in opaque white and bright emission as transparent black.
Generally speaking, observational data tends to have a background value of 0.0, so using this for the lowest data level is usually a sensible choice. Although the data calibration can mean that some noise values are below zero, using negative values in the display settings tends to show so much noise that the display is overwhelmed. This can also make it difficult to view the faintest sources. For these, it's better to use 2D mode, which has no such problems with data display.
"Colour scheme" : this drop-down menu chooses the colour scheme to use when loading the data. This uses a variety of matplotlib standards plus a few custom sets. This cannot be changed after the data is loaded without reloading the data.
Not all colour schemes are generally suited for this sort of display. In those cases, a greyscale map is used for the transparency and the specified colour map is only used for the colours. For more information, see the "Advanced file options" panel and its help file.
Note that the default values are usually fairly sensible. Starting with a simple greyscale colour scheme is highly recommended before trying anything more complex ! The basic operational procedure is :
- Choose data level percentages, press "Get levels", adjust the percentages or actual values as needed
- Choose whether a linear or logarithmic colour transfer function would look better
- Choose the colour scheme
- In the next panel, press "Load data" - if necessary, adjust the settings until happy or bored
If you alter any settings in this panel, you will have to reload the data to see the changes. However, some alterations can be made in real time in the "Loading and display controls" panel.
Loading and display controls
This panel lets you actually load the data, as well as make some simple alterations to the display afterwards.
"Load data" is the main button to load the data with all the specified parameters, including those which are set in other panels. This will also write the FITS header in a human-readable text file ("FitsHeader.txt") and draw the axes of the data, using a world coordinate system if possible. The spectral axis of the coordinates can be changed in the "FITS Analysis" tab, "Spectral coordinates" panel.
NOTE : It's strongly encouraged to have the console window visible while running this to check for error messages. In the case that FRELLED cannot find any images it needs to load, it will report this (along with instructions about what to do) to the console, but annoyingly, the popup error that should alert the user simply refuses to work. Errors also cause the view of the FITS file to be automatically disabled to prevent confusion. The user has to manually re-enable this ("Analysis" tab, "Files" panel) when re-loading the data. All this and more is described in the console.
"Auto import" will set some sensible defaults and load the data file with one button. Very useful if things are a bit daunting !
"Quick preview" will load the data using the user's specifications, but only along the shortest axis of the cube. This means the data will only be visible from a limited range of viewing angles, but loads much more quickly than when using the full data set.
"Opacity" sets the maximum opacity level of the data, where 0 is transparent and 1 is opaque. All data points at or above the maximum specified data range will correspond to this value. This value is set with a slider, and can be adjusted after the data is loaded.
"Clip alpha" forces everything below the specified opacity value to be completely transparent - this is purely clipping, not scaling. Useful for removing excessive noise. Can be adjusted after the data is loaded.
The colour picker (default white) box sets the base colour of the material. While setting the actual colour transfer function (via the drop-down menu in the "Main display settings" panel) requires reloading the data, this setting can be altered after loading the data. This is quite useful if the data uses a greyscale colour scheme, but results can be more complicated if using something else.
When changing the base colour, use the "Apply all changes" button in the "Help and changes" panel at the bottom, or just change the opacity value as described above.
Advanced file options
This panel is closed by default to prevent scaring new users, and most of its functionality isn't routinely needed. However, it allows for powerful control of the display (at the loading phase) when necessary. For more controls over the display after loading, see the "Advanced display options" panel.
FRELLED works by converting the FITS data into a Blender-readable sequence of PNG images, one for each slice of the data (i.e. each spectral channel). Two sets of XY/XZ/ZY toggle buttons in this panel control how to process the file. These follow the kvis convention : XY=RA-Dec, XZ=RA-Vel, ZY=Dec-Vel in a typical observational data file.
The first set of projection toggles, the "export" buttons, set which of these projections should be used when converting the file to a FRELLED-compatible sequence of PNG images. By default all three are used, so that the user can see the whole data set from any angle. For larger files this can be a slow process, however, so having the option to disable these can give much faster loading times. Similarly, the "import" buttons control which projection, once exported, will be actually shown in FRELLED. This doesn't affect the loading speed as much as the export buttons, but it can still be significant. If you know the data has already been exported in a format you like, disable the export buttons to save yourself a good deal of time.
Since you can set the export and import buttons independently, you can in principle display each projection with different colour settings. To do this, disable all the import buttons and export each projection one at a time (e.g. set the data display range, set the XY projection, press "Load data", and repeat with the other projections). Then disable all the export buttons and enable all the import buttons. Granted, this is a very niche feature, but it's nice to have.
The "Diff. cols" checkbox selects a different data range (using the adjacent sliders) for the colour values than the opacity values. By default this is false.
Note that each FRELLED map consists of a sequence of PNG images, each containing colour and opacity information. The colour is set based on the colour scheme chosen by the user, but the opacity information is generally generated by a greyscale map regardless of the user's choice (with a few innately suitable exceptions). This ensures that the transparency always varies between completely transparent and completely opaque. If this setting is changed, the data must be reloaded.
One notable use of this feature is to choose a colour scheme where the lowest data value would be shown as black. Using this option and setting the lowest colour value to a higher data value than for transparency, the faintest data will now be shown as white (or rather, the value of the colour set in the base material colour picker) instead, making it much more visible. Remember, as described in the "Main display settings" panel, setting a minimum value greater than the maximum reverses the transfer function, i.e. reversing the colours in this case.
The "Log cols" checkbox allows a logarithmic colour transfer function for the colours. In combination with "Diff. cols", this means one can have a linear transparency opacity transfer function but a logarithmic colour transfer function, or vice-versa depending on the settings. If this setting is changed, the data must be reloaded.
"Viewport shading mode" sets how to display the data in the 3D window, and can be altered after loading the data :
- Textured : The default view which FRELLED is designed for and almost always the correct choice.
- Box : Show everything as cuboids based on their maximum extent. Essentially only useful for debugging.
- Wireframe : Show everything based on its edges. Can be useful for isosurfaces.
- Plain surfaces : Show the faces of every 3D object as smooth grey. Again useful for isosurfaces but not much else.
- Material : Basically the same as Textured, included for completeness.
- Rendered : NOT RECOMMENDED ! Shows the data using Blender's render engine. Much more powerful than the default view, but slower than a tranquilised snail that's been run over by a tortoise riding a sloth. However it can be useful for displaying isosurfaces (see also the video guide), provided the volumetric display has been disabled.
"Stretch to cube" : forces the Z-axis to be shown at the same size as the X-axis.
Conventionally, the Z-axis in observational data is the spectral axis, which has a completely different nature to the spatial component. Thus, sometimes cubes can have far more or fewer channels than they do spatial pixels, so the data could appear elongated or compressed. Forcing them to appear of equal size can make for a much better display, though sometimes it's better to disable this.
If this setting is changed, the data must be reloaded. However, if you've already loaded the data and this is the only change you want to make, remember you can disable the Export toggle buttons. This will make reloading the data many times faster.
"Sparse factor" : in 3D mode, the data is shown as the sum of opacity values along the line of sight. Therefore, if the data isn't a perfect cube, it will always appear more opaque from one orientation than other, since there will be a different number of slices in each projection. To overcome this, this factor ensures that only every nth slice of the data is rendered and imported, such that all three projections have approximately the same number of visible slices. This has two effects. First, since Blender's colour precision is rather limited, it generally makes for a much higher quality display when using large files (yes, really, showing less data gives a much better appearance !). Second, by reducing the number of slices used, it greatly decreases loading times. It's usually worth leaving this at the default value. The settings are as follows :
0 = disable sparse sampling and load every slice in every projection loaded
1 = default, attempt to equalise to the shortest projection (best balance of speed and quality)
n = load every nth slice, scaled by the shortest projection - useful for quickly checking displaying settings
Velocity colour map
This panel is hidden by default because it is not used routinely. It enables you to replace the flux-based colours with colours from the spectral (Z) axis of the cube. This can be useful for observational data, especially when making 2D figures of 3D information.
By default, velocity colour maps are not generated along with the flux maps. Click "Enable" in this panel for this to work. Choose a colour scheme from the drop-down menu on the right.
To avoid having to press the main "Load data" button, which would regenerate the flux maps as well, simply press "Update velocity maps". Any existing flux maps are not altered - only the display in FRELLED is changed, replacing the colour component with that from the velocity maps once they are generated. To remove the velocity maps, uncheck the "enable" setting and press "Update" again.
Multi-volume display settings
This panel is hidden by default because it's scary enough to give new users nightmares, and is only useful in very specialised cases. But if you do have such a case, it's a lot of fun.
"Mulit-volume display settings" panel
It's possible to combine information from two different FITS files and display the results in different ways. Both files must have IDENTICAL pixel dimensions. They do not necessarily require the same world coordinate systems, but this is highly recommended.
The "Use second FITS file" checkbox selects whether or not to use the different FITS file selected in the file browser below. A primary FITS file must always be selected in the "Main display settings panel" - the secondary FITS file cannot be used independently.
"Colour/Volume" toggle - this selects how to incorporate the second data, which can be done in two different ways :
- "Colour" - in this mode, the primary FITS file provides the transparency information while the secondary file provides the colour maps. For example, with simulation data, one could use the primary file to display the density information as opacity, with the colour set by a different component from the same simulation, e.g., temperature. In this mode, the data MUST be loaded using the main "Load data" button in the "Loading and display controls" panel - NOT the "Load second volume" button here.
- "Volume" - this mode shows the secondary FITS file to provide an independent volumetric overlay, e.g., if you have two different chemical species (HI and CO, for instance), you can display them using fully independent density and colour information. In this case, data can be loaded EITHER using the main "Load data" button in the "Loading and display controls" panel, or with the "Load second volume" button here. The latter is recommended : it will avoid re-rendering the primary FITS data, making it much faster to load the second data set.
- In both cases, the projections to export/import are set in the "Advanced file options" panel, which affect both files.
Note : If you load in a file in Colour mode, and then load in a file in Volume mode using "Load second volume", this will overlay the file using the current display. That is, the primary volume file will comprise alpha and colour information from the two files already specified, with the new secondary file effectively overlaid as a third data set. Use the main "Load data" button to restore the expected behaviour of two separate data volumes.
All the remaining options in this panel are as in the "Main display settings" and "Advanced file options" panels - they set the data range to display, whether to use a logarithmic CTF etc., only applied to the secondary FITS file instead of the primary.
IMPORTANT : By default, FRELLED is configured for displaying a single FITS file, and using these settings won't give nice results for multiple files. The best way to get started is to go to the "Advanced display options" panel and set all opacity/colour toggles to "Add". See the next section for more information, or even better, watch the video guide. Finding settings that give good results with multi-volume data is probably the most complex operation in FRELLED so I'd strongly recommend watching the tutorial.
Advanced display options
Another panel hidden because most of the time the settings here can be safely ignored. Extremely useful for multi-volume display, however.
Here the user can set a variety of display options to alter the appearance of the data. All of these settings can be applied after the data has been loaded. It may be necessary to use the "Apply all changes" button in the "Help and changes" panel to ensure the changes are applied correctly.
FRELLED is designed to set sensible default values for the most common usage conditions. The options here are intended for more advanced users. While they offer a great deal of control over the display, each option can interact with multiple other parameters (including those set in other panels), making the results sometimes unpredictable.
The left and right columns set identical parameters for the primary (left) and secondary (right, if loaded) FITS files.
NOTE : the rest of the options always work for the primary file, but only affect the secondary file if loaded using "Volume" mode !
Opacity/colour options :
- "Mix/Multiply/Add" - set how the opacity or colour map should affect the base property, set from either the "opacity" slider or the colour picker in the "Loading and display controls" panel.
- "Mix" essentially replaces the values with the map and ignores the base property.
- "Multiply" takes the base value and multiplies it by the map value. If this mode is used, the opacity slider cannot be zero or nothing will be visible. Conversely a value of 1 does not guarantee everything will be opaque either, depending on the map. Note also that the base colour value should not be black, otherwise again everything will be black.
- "Add" adds the values of the map and base colour together. Unless the base material value is very low, this can easily give saturation effects.
Sliders :
These set the contributions to transparency and colour from each of the FITS files. If zero, the component has no effect, e.g. the opacity value will be constant and equal to the value in the slider in the "Loading and display controls" panel. If 1, the contribution will be maximised.
These sliders update in real time, however this can be slow - it may be better to click inside the slider to type a value numerically.
COMMON USAGE SETTINGS THAT GIVE NICE RESULTS :
- One single FITS file, or using the second file to affect colour : Opacity 0.1 (must be > 0.0), clip alpha 0.0, base colour white, all advanced display options to Multiply and 1.0
- Multi-volume mode : Opacity 0.0, clip alpha 0.0, base colour black, all advanced display options to Add with numerical values adjusted according to the particular files
Heightfield display
This one is hidden by default as it can only be used in 2D mode.
This panel lets the data be shown with displacement according to the flux value. In each slice, each pixel will be shown with the same colour value as it normally would, but will also be displaced along the perpendicular axis according to its data value. This is a useful alternative display that can mitigate the problems of choosing a sensible colour scheme and data range.
When enabled, heightfields update according to the current frame, i.e. the slice of the data currently visible (note that this can be slow, as the heightfield is shown with the same spatial pixel resolution as the data). Ordinarily, rotating the view will change the projection shown, but with heightfields enabled the view is locked to the current projection. Disabling heightfields restores the usual behaviour.
At present, the displacement happens by an essentially arbitrary amount so as to fit on the screen. It provides visual cues as to the distribution of the data values, but one cannot use the displacement to determine absolute values.
Heightfields (also known as height maps or displacement maps) are a good way to deal with data of high dynamical range. Even using a logarithmic transfer function sometimes isn't enough to prevent saturation of the highest flux levels while ensuring a good display of the faintest features. With height maps there is no such problem; different dynamic scales are seen, in effect, by zooming in and out.
"Scaling" : Set the scaling function used. Default is linear.
"View" : Controls the display method Blender uses. In this context, "Textured" will show heightfields with the colours already generated, while "plain surfaces" will show everything as plain grey.
"Scale factor" : Numerical multiplier for the scaling of the displacement.
"Generate/enable" : Switch on heightfields. Note that these can be slow for large data sets, and updating the current slice is also slow.
Help and changes
This panel lets you access this document and also save your display settings.
"Apply all changes" ensures all the display settings chosen are actually applied. Generally only needed if you alter the material base colour.
The "Save", "Load", and "File selector" boxes let you save all the display settings to a text file. This is useful if you want to apply these settings inside another FRELLED.blend file.
VERY IMPORTANT : The display file saves the absolute file paths to the FITS files, but FRELLED can get quite upset if these aren't in the same directory as the FRELLED.blend file. If you copy a display file to a new FRELLED directory, be sure to also copy the FITS files you want to use to that same directory. Then, after loading the display file in FRELLED, use the GUI file selectors to manually select the FITS files in the new, correct location.
FITS Analysis tab
From this tab you can use FRELLED to analyse your data set in different ways. You can transform the spectral coordinates into different velocity conventions, navigate the world coordinates of selected features, pick out the brightest points, and analyse particular sections. The principal basis of this is creating objects called regions (sometimes also masks or even mask regions). These are volumes over which the various analysis features can be employed, e.g. computing total flux, plotting contours, etc.
Since FRELLED was initially designed for HI data, it also includes a dedicated section that allows you to interface with miriad's mbspect task for analysing spectra. In principle I see no reason this won't work with other spectral lines, but I'm far less familiar with the pitfalls an nuances of these so I haven't tried to make this robust.
Files
Here you can select a different FITS file from the one displayed to use in the analysis. This file will be used for all other operations in this tab, including calculating world coordinates, generating contour plots and other maps (see the "Region mapping" panel), and by default used for miriad tasks, though this latter can be specifically overridden.
Use of a different FITS file for analysis than visualisation can be helpful. Sometimes you need to process the data to show features more clearly, but you want to operate on the original data values. Additionally, this option gives contour plots and renzograms another way to display multi-volume data : you can show one FITS file volumetrically and overplot the contours of another. In principle, this means you can visualise an unlimited number of different data sets all at once.
You can also here toggle the display of the file via the "Show FITS file" button. This is also possible by altering the various display options in the Display tab, but the toggle button is more convenient - especially when using other features in the "Analysis" section.
Spectral coordinates
This panel is hidden by default as FRELLED normally chooses something sensible for the spectral axis coordinates, at least if you're using HI data.
By default, FRELLED inspects the FITS header for popular incarnations of the keyword that provides the rest frequency (e.g. RESTFREQ, RESTFRQ, FREQ0). The GUI values will be updated to show this if found. If no matching header keyword is found, FRELLED uses the values shown in the GUI, i.e. it defaults to assuming HI. A drop-down menu presents a couple of other spectral lines as well as access to the "Splatologue" website which has just about every rest frequency known. The user can also enter a frequency manually in the text box - "Hz", "kHz", "GHz" etc. are all understood as frequency units.
The lower drop-down menu sets the velocity convention to use. The default is to use that specified by the FITS header, if any. Other options are (following the miriad naming conventions) :
- FELO : optical velocity
- VELO : radio velocity convention
- FREQ : use actual frequency instead of velocity
- Relativistic : true relatavistic velocity
- Pixels : just use the image pixel numbers, i.e. channels
Further details on these can be found here.
"Hide" toggles whether the main axes (of the whole cube) are visible or not. This is useful in viewing the data as clearly as possible, and also when making animations and figures. This also toggles the display of any regions created, though not their associated maps (see "Region setup" and "Region mapping" panels).
"Redraw" redraws the main axes using the specified settings. This only applies to the main axes shown around the whole FITS file, not axes around individual regions within it (again see the "Region setup" and "Region mapping" panels).
Navigation
This panel gives information about the current coordinates of the 3D cursor. IMPORTANT : the actual cursor position in FRELLED and the values shown in the GUI are necessarily independent and the two do not update each other automatically - the user must take explicit action to ensure the two correspond with each other.
The cursor can be placed manually by left-clicking in the 3D window. To ensure it's in the correct place in all three dimensions, change the viewpoint (rotate by holding down the mouse wheel or using the buttons described below) and update its position again. The view is changed only across the plane of the screen. A good practise is to make use of the three major viewpoints (XY, XZ and ZY projections, see "Navigating in FRELLED" and below) in orthographic perspective, guaranteeing the cursor is placed exactly where you think it's placed.
"Get sky coords" attempts to use the astropy name resolver to set the coordinates of a source specified in the associated text box. At present this will only set the sky coordinates, not the spectral coordinates. Coordinates are set in the boxes below, but the 3D cursor position is not updated automatically.
The "RA" and "Dec" boxes set the sky coordinates in sexigesimal format, which will automatically update the corresponding decimal format boxes below (and vice-versa). Galactic coordinates, using latitude and longitude, are also supported, and choosing the coordinate system toggle will automatically convert the current values. Updating these values does not automatically update the cursor position.
"Velocity" sets the spectral axis coordinate using the specified convention (from the "Spectral coordinates" panel), generally velocity in km/s but other formats are also supported. Updating this value does not automatically update the cursor position.
"Pixel coordinate" boxes control the cursor position interactively. Updating these alters the cursor in the 3D viewport but does not automatically recalculate the world coordinates in the GUI.
"Where Am I ?" updates the above world coordinate GUI boxes with the values at the current cursor position. Note this has some additional functionality. The cursor coordinates are printed to the terminal so you can copy and paste them more easily, but it also allows you to find the angular distance between objects. The angular separation (if the cube has a compatible WCS) between the cursor and all selected, visible objects will be printed to the terminal in decimal and sexagesimal degrees. The cursor position is then reset to the current pixel coordinates. This may be slightly inaccurate since the true cursor position is a real number whereas these are integers - use "Take Me There !" to restore the accurate position.
"Take Me There !" moves the cursor to the world coordinates specified in the GUI boxes.
"Centre cursor" is useful if the cursor gets lost, repositioning it in the centre of the cube. The GUI coordinates are not updated.
"Peak cursor" places the cursor at the peak flux value along the line of the nearest line-of-sight projection. That is, if viewing the cube face-on, it will set only the spectral coordinate of the cursor; if viewed not quite face-on, the same is true. At other angles it may set the RA or Declination instead. This is very handy when trying to position the cursor on bright features of interest.
"Ortho" toggles the display between accounting for the effects of perspective or not. Using perspective can make viewing details in the data much easier, but can be confusing when making figures and positioning the cursor. This can also be toggled using "5" on the numeric keypad.
"XY / XZ / ZY" buttons sets the viewing angle to the major projections, following the kvis naming schemes. In 2D mode this has extra functionality :
- If no or multiple regions (see the "Region setup" panel) are selected, it will also set the visible slice to the centre of the cube in that projection.
- If a single region is selected, it will go to the slice corresponding to the centre of that region. This helps make it easier to adjust the size of a region in 2D mode.
See "Navigating in FRELLED" for information on using the numeric keypad to navigate.
Region setup
This is where things get interesting. This panel enables the user to define regions of various shapes within the data. These can be used for analysis of specific parts of the data, e.g. by computing the total flux present or generating maps of individual structures. They can also be used for hiding and cataloguing sources. Regions can be generated within FRELLED but also saved and imported using an external file.
Regions are named according to the the syntax "Prefix_00N" where the prefix is specified in the text box (default, "Name" - PLEASE change this to something more interesting !) and the counter in the adjacent numerical slider. The counter starts from 1. New regions are added with the "Add" button. If a region of the same full exact name already exists, the counter will be incremented by 1 until an empty slot exists. E.g. if the user tries to create Galaxy_005 but this already exists, FRELLED will try to create Galaxy_006, 007 etc. until there is no match with the existing regions.
The drop-down menu selects the shape for new regions. Note that most analysis functionality assumes cuboid regions, and will not give the correct result if other shapes are used !
A special option is available in the the menu :"WholeCube" - this first places the cursor at the centre of the cube and then creates a region of the exact size of the entire FITS data set.
The "rename" button attempts to change the prefix of the selected regions, but retains the counter value unless this conflicts with an existing region. In that case, as with adding new regions, it will attempt to search for an empty slot in the sequence and select a new count value accordingly.
The "Add" button creates a new region of the specified shape and (if possible) name, at the current 3D cursor position (see "Navigation" panel). Regions are added to both 3D and 2D modes, so when you switch from one to the other, the same regions will remain visible.
The "Delete" button deletes all selected regions and their associated maps. This includes checks to ensure that only selected REGIONS are deleted and not any other objects.
The "Import" and "Export" buttons allow the user to save and load the selected regions to external files. Regions are saved in both image pixel coordinates ("BlenderMask.txt") and world coordinates ("WCSMask.txt"). Both currently assume cuboid, non-rotated regions. Only the BlenderMask file correctly preserves the exact size of the regions. By default, the Import button will opt for BlenderMask if both files exist. Ensure that this file is deleted or renamed if you need to force it to use the WCS file.
Both mask files are given the extension _00N, where N is a numerical counter increased according to the files already present. This prevents exporting regions from ever overwriting any files. In addition, a file will only be written for export if at least one region is selected. Importing regions always uses the file with the highest counter value, i.e. the latest file written.
A powerful use of the Import button is to load sources found outside FRELLED. The syntax for the region files is as follows :
1) BlenderMask :
name, x, y, z, sx, sy, sz
Where all variables are whitespace separated. "Name" is the ID of the source. "X, y, z" are the pixel coordinates of the source (normally referring to the RA, velocity, and Declination axes). "Sx, sy, sz" refer to the pixel dimensions of the source along the corresponding axes.
2) WCSMask :
name, ra, dec, vel, velw
Where "ra, dec" are in decimal degrees. "Vel" is the systemic velocity of the source and "velw" its velocity width in km/s.
Both files can optionally use a header line denoted with a hashtag.
The "(Un)select" button toggles the selection status of selected regions, based on the specified name and numerical count. For example, suppose you have all regions called "Galaxy_XXX" currently selected. If you set the prefix in the text box to "Galaxy" and the count to 4, then pressing this button will toggle the selection status of Galaxy_004. If you use 0 for the count, it will apply to all objects with the "Galaxy" prefix.
If you have multiple sets of objects, e.g. one called "Galaxy" and another called, "Wombat", then if you change the prefix, the selection status of the original objects is not affected. For example, if you begin with no objects selected at all, then specify "Galaxy" and "0", all "Galaxy" objects will be selected. If you then just change the prefix to "Wombat", then all the "Wombat" objects will also be selected.
The colour box sets the colour to display for the selected regions in the 3D view (default black).
"Wire/Solid/Hybrid" buttons sets how to display the selected regions :
- "Wire" : use a transparent wireframe. Useful for highlighting a particular part of the data whilst still keeping it fully visible.
- "Solid" : show as a totally opaque box, with the selection outline hidden. Useful for hiding distracting features, e.g. when cataloguing sources, but it can be difficult to know which objects are selected since their outline is not visible !
- "Hybrid" : show an opaque box with the complete wireframe outline. In this way the full extent of the source is visible but the data within the boundary is hidden.
A brief guide to working with regions
Regions can be selected with the GUI controls described above but also by right-clicking on any part of them. To select multiple regions (and/or unselect regions already selected) right click while holding down shift. To (un)select all regions, either : use "0" in the counter; or place the mouse pointer in the 3D window and press A on the keyboard (note that this may select other objects as well).
Regions exist only within Blender and do not alter the FITS file in any way.
Blender's objects differentiate between a selection and active status. Both are highlighted with an outline. The active object (the last object selected) is given a different colour. There can only ever be one object active at any time, but there is no limit on the number that can be selected.
The outline colour visibility depends on the selection status and the display mode for each region :
- Wireframe objects will be white when unselected, blue when selected, and light blue when active.
- Solid objects will have a white outline (blocked by their internal opaque region) when unselected, which is not visible when they are selected or active.
- Hybrid objects have the same appearance as wirefame objects regarding their selection status, except that the data within them is hidden from view.
Important : At the lower left of the 3D viewport, the name of the currently active object is shown in white. Selecting a region with the GUI controls described above does not automatically make it active, so the name in the viewport may not change using these buttons. However, the right click method does automatically change the active object.
To move a region, select it. Ensure the mouse pointer is within the 3D viewing window. Press G (for "grab") on the keyboard (press once, do not hold). The region can then be moved across the plane of the screen using the mouse. Left click to accept its new position when happy, or right click to cancel and restore its original position.
Regions can be resized in much the same way using the S key ("scale").
After pressing either G or S, the user can also press X/Y/Z on the keyboard to constrain the movement or scaling to a single axis. As well as using the mouse, the user can also then type numerical values on the keyboard to be exact, e.g. "G", "X", "-5" will move the region 5 pixels to the left.
It's also possible to rotate objects with the R key. However, this is not currently well supported in FRELLED. This should be used ONLY for masking features and nothing much else (well, maybe summing flux is okay - see the end of the "Region mapping" section).
To move an existing region to an exact world coordinate, enter the coordinates in the Navigation panel and use the "Take me there !" button to position the 3D cursor. With the region selected (ensure only one region is selected !), press shift+S and choose "Selection to cursor".
Region mapping
This panel is hidden by default because it is very, very scary.
Once regions have been created, this panel lets the user create various maps and contour plots to help analyse the features present. Recall that the display of the FITS file can be toggled with the "Show FITS file" button in the "Files" panel, which can make it much easier to see the generated maps. Similarly, make sure the regions are set to "wire" display mode using the "Region setup" panel above - by default this is enabled whenever new maps are created, but some operations may reset this, especially in 2D mode.
The top row of numerical values set parameters for the maps. From left to right, they set :
- The minimum data value or contour level to display
- The increment (step interval) for any contours to display
- The maximum data value or contour level to display
- The thickness to use when rendering contours (can be updated after contours are generated, if they are selected first)
At the end, the x/Logs toggle button enables using a geometric progression for contours or, equivalently, a logarithmic colour transfer function for maps. By default this is inactive and linear scales will be used. When active, the contour increment is treated as a multiplier. For example, if the user specifies a minimum value of 2.0, a step of 3.0 and a maximum of 50, the contours generated will be at the levels : 2, 6, 18.
The "Map data scaling" toggles the numerical values for the contours and maps, between using actual values of the data (e.g. flux) or whether the user's entries refer to the normalised values within the region : a minimum value of 0.1 would now correspond to 10% of the peak value in that region or map. Note that the actual flux values plotted when using contours are always printed to the terminal, so you'll always know the real values and not just the percentage levels.
The "Map display" button toggles the selected map between transparent and opaque. By default these are opaque, which are brighter. Transparent maps mean you can see multiple maps at once, e.g. overlay flux and SDSS images. The toggle button affects both maps which subsequently generated and those which are currently selected. In the latter case, the map object itself must be selected, see the "Operations" section in this panel (described below).
"Map type and colours" section - from left to right :
- Drop-down menu of the map type to produce. Options are :
- Integrated flux map (moment 0, sum of flux values along each spectra)
- Peak flux map
- Velocity map (moment 1)
- Velocity dispersion map (moment 2)
- Integrated flux contours : moment 0 but contours instead of a map
- Peak flux contours
- Renzogram : contours at a fixed flux level, one per channel (uses the minimum contour value), coloured according to the velocity
- Isosurfaces : full, 3D surfaces at a constant flux level
- SDSS RGB : shows the SDSS RGB images for the field of view of the specified region
- Axes : plot axes around the selected region, useful for making figures
- Drop-down menu of the colour scheme to use for the map or contours
- Drop-down menu of the velocity range to use when computing renzograms or integrated flux contours. Options are :
- Global : colours are defined by their velocity within the whole data range
- Local : colours are set based on the position in velocity within the selected region
- Selected : colours are set based on the position in velocity over the range of all the selected regions
- Colour picker : when using the special "solid" colour scheme, uses a fixed specified colour.
Note that because of the limitations of Blender's layer system, a couple of maps cannot be shown at the same time : peak and integrated flux maps, and peak and integrated flux contours.
Renzograms are quite straightforward : one contour per channel, coloured as described above. The increment, maximum value and step all have no effect on renzograms, only the minimum value is used.
Integrated flux contours are each coloured according to the systemic (central) velocity of each selected region, with their fractional position computed as described above.
Isosurfaces have some special behaviours. By default they are transparent, and the user has some control over their opacity levels. The lowest flux surface has its opacity determined by the "contour thickness" parameter, while the highest flux surface is constrained to always have an opacity of 1.0. Intermediate surfaces are linearly interpolated between the two extremes. Like the volumetric FITS data, isosurface visibility can be controlled using the "Clip Alpha" slider in the Display tab. If multiple isosurfaces are present, then they have no other functionality. However if only a single surface is present, then its opacity is controlled with the same "opacity" slider in the main Display tab that's used to control the volumetric data, with the colour picker below it used to set the surface's colour.
Although by default isosurfaces are transparent, Blender's transparency gives extremely ugly artifacts on curved surfaces. There are several workarounds for this :
- The preferred option is to change to the "plain surfaces" display mode from the "Advanced file options" panel. Using the clip alpha slider will allow you to see all the surfaces, just not at the same time. Be sure to disable the view of the volumetric FITS file for this !
- Setting the display mode to "Rendered", as in option 1 keeping the FITS file view disabled. This can give a decent result but doesn't update instantly as you change the view.
- Export the isosurfaces to Cycles/EVEE materials for viewing in Blender 2.9, see the "Virtual Reality" section. This is a fairly drastic step, but it does work.
More information on moments can be found here.
"Operations" :
- "Generate" produces the specified map or contour plot. Note that these are generated as Blender-internal objects, which the user can select and manipulate as they would regions.
- "Select" allows the user to select the chosen maps (as specified in the drop-down menu) for the selected regions, unselecting the region itself. Note that most operations require only the region to be selected, but updating contour thickness of existing contours requires the contour objects themselves to be selected instead.
- "Delete" deletes the specified map.
"Toggle maps" is a series of toggle buttons to allow the display of all the various possible maps. When the user generates any, the relevant toggle is automatically activated. Note that "Annotations" and "Cameras" refer to components generated in the Figures tab. Unfortunately the number of visibility layers in Blender is limited, so some maps share layers (peak and integrated flux maps, and peak and integrated flux contours). If both of these have been generated, the one will obscure the view of the other. The only solution is to delete one of the pair, or make them transparent.
"Point clouds" enables the display of the vertices computed when running the SumFlux task described below.
"Sum flux" : these buttons attempt to calculate the total flux present within a specified region and apply beam corrections to the result. All results are printed to the terminal.
- "Cubes" is a fast, reliable estimation which is 100% accurate provided the region is a regular non-rotated cuboid.
- "Arbitrary" is a semi-experimental method that computes the flux within arbitrary volumes, e.g. cones, spheres, monkeys, whatever. This slower method generates and displays a vertex point cloud representing the data used in the summation. The user can edit this manually (though this requires Blender expertise which is beyond the scope of this guide) and use the "recalculate" button to update the result.
Query external databases
This is hidden by default for no particular reason.
This allows the user to run some basic queries from NED (in either the classic or modern interfaces) and the SDSS, depending on what is currently selected :
- If no regions are selected, the query is done at the position of the 3D cursor
- If one region is selected, the query is done at the position of the object and the result is opened automatically in a web browser
- If multiple regions are selected, an HTML list of queries for each individual object is generated, and shown in a web browser
The checkboxes let the user choose (for NED queries) to include only galaxies and/or only objects with known redshift measurements. The SDSS query ignores the checkboxes; it simply retrieves images and opens the SDSS Navigate tool for the selected regions.
"Query FOV" sets the size of the query in arcminutes. "0" uses the X/Y average size of the region instead.
Miriad operations
This is hidden by default because this is designed for specialist cases. Installing miriad can be a bit of a procedure. Installation instructions for Linux and Mac can be found here. For Windows a native installation is not possible, but under Windows Subsystem for Linux the procedure is identical to Linux. A complete guide to installing WSL can be found here (it may be outdated since WSL can now be retrieved from the Microsoft Store, so it might be easier now).
This panel allows the user to use some miriad tasks interactively. This guide assumes the user is already familiar with miriad and only describes their operation within FRELLED. Most operations produce text files containing the inputs needed for miriad, so it does not matter if miriad is actually installed or not (special provision is made for this in the case of mbspect, where the tasks will run interactively from FRELLED).
The file selector allows the user to specify a miriad file as most miriad operations must be run on files of miriad's own format. Alternatively, specifying any FITS file here will enable an automatic conversion of the FITS file to miriad's own format when the mbspect operations are run, provided such a file does not already exist (i.e. the same name but with a ".mir" extension). If it does exist, the miriad operations will simply use that file. The FITS file itself is retained - no files are overwirtten by this process.
Note that specifying actual miriad files here is a bit finicky, as miriad's own "file" format is actually a directory, while the Blender browser only lets you select files. The workaround is to use the browser to navigate to the correct location, type in the name of a file (even one that doesn't exist will do), press "Open file" in the file browser to select it, and then manually edit the text box to specify the correct name. The easiest option is usually to choose the FITS file you're visualising/analysing. As above, this will be automatically converted and the miriad file given a ".mir" extension.
"Mask" generates the miriad task "immask" for the selected regions, producing a single file "MiriadMask.txt" containing all the input parameters. The user can then copy and paste this into miriad to run it. This is not run in miriad automatically.
"Make FITS" is very similar to "Mask", but uses the "FITS" task instead, enabling conversion of individual regions to the miriad format. Note that miriad only allows the use of regions when converting from a miriad file into a FITS file, not the other way around. Again this is not run in miriad automatically.
"Mbspect spectral analysis" runs the "mbspect" task, which analyses individual spectra. All parameters are as described in the miriad documentation. Using "Auto" will attempt to automatically choose sensible values or simply leave the parameters unset.
FRELLED will determine if the user has miriad installed or not (including via Windows Subsystem for Linux). If they do not, it will only produce text files of the miriad input parameters instead. If they do, it will run mbspect interactively provided only one region is selected, or produce text files if multiple regions are selected.
"Miriad quick" runs mbspect interactively, overriding the plot setting to force it to show the spectrum in an X-window (as long as only one region is selected). This means you can easily check which settings work and adjust them (and/or the region itself) if needed. This is especially useful for determining a sensible velocity range, allowing the user to simply scale the region rather than manually inputting values. The output values are shown in the terminal
"Miriad full" is designed for when the user is satisfied that all the parameters are correct. It produces text files each containing the input parameters, the miriad measurements, and a copy of the spectra. It also overrides the user's setting and forces the production of a postscript graphical output file of the spectra.
Both operations of mbspect produce text files in the "Mbspect" directory. The files containing only the input parameters always contains the user's choice of device (postscript or X-serve from the dropdown menu), but the full log file (only produced in "Full" mode) insists on using postscript.
Note that the mask range is set to ALWAYS include the profile range, regardless of whether its Auto toggle is enabled or not.
FITS Figures tab
This tab lets you control how to make nice pretty plots and animations you can use to impress people. You can do this for individual regions or the whole data set. It contains the tools to annotate the file (add labels and other features) and set up animations; to control the display of the FITS file itself and its axes, you'll need to explore the other tabs.
Main display options
This controls the global display options in FRELLED, useful for ensuring the figures are appropriate for screen and/or print. The drop-down menu allows the user to select preset options for the colours which are suitable for either case. Users can save their chosen settings to a file for retrieval later. IMPORTANT : when reloading a file, FRELLED may revert to the default colours, causing the appearance of missing objects, no visible active object name in the lower-left corner of the viewport, and other artifacts. To fix this, just choose an option from the drop-down menu and this will refresh the colours.
The "background" colour picker chooses the colour for the 3D background display. The "grid" toggle shows or disables a 3D grid in the background, with a corresponding colour picker.
The "Selectable" button sets whether the selected region axes are selectable or not. By default they are not, so that the user can't accidentally edit them. If enabled, the user can manually adjust them, including the text of the labels (select the text, tab key to enter edit mode, type as normal, tab again to save changes and return to object mode). The region of the axes you want to edit must be selected for this to work ! This is especially important when turning this off : if you have the axes selected (and not the corresponding region) and deactivate the toggle, they will remain both selected and selectable.
The final colour picker and file selector apply to the fonts, allowing the user to manually choose a font and colour for the axes and their labels. So yes, you can use Comic Sans if you really want to, but I hereby absolve myself of any responsibility on that front.
Region annotations
This lets the user add text and other labels, for creating shiny figures.
The drop-down menu specifies the type of annotation to create. Available options are :
- Text - creates the text using the entry in the text box below
- Beam - checks the FitsHeader.txt for the BMAJ and/or BMIN (beam major and minor axes) parameters. If found, creates an ellipse of the corresponding beam size, otherwise warns the user and assumes a default size of 5 pixels
- Beam line - as above but just draws a line of the length of the major axis
- Circle/torus - makes a round thing
- Box - makes a square thing
- Cross - keeps vampires at bay
- Plus - keeps mathematically-inclined vampires at bay
- Arrow - a pointy thing
The colour picker sets the colour of the new or already-selected annotation.
"Scale" set the size of the feature.
"Hide", when toggled, hides the selected annotations. Unsetting this reveals all currently hidden annotations.
"Delete" deletes the selected annotations. Quelle surprise.
"Add" generates the chosen feature at the position of the 3D cursor. Note that you must have a single region selected for this to work - all annotations need to be associated with a region. If you don't need regions, use the special "WholeCube" region.
Region/figure management
When you have many regions each containing maps and annotations, the display can quickly become confusing ! This is especially true if the regions are overlapping. The buttons in this panel let you quickly focus on one specific region, hiding all of the other regions and their components (maps, axes, annotations - everything) from view. Or you can do the reverse and hide the selected regions plus their components, leaving the rest alone. The specific operations are :
"Focus on selected" hides all regions and all their components except for the selected region(s). While you can do this with the toggle buttons in the "Region mapping" panel of the Analysis menu, this is more convenient since it not only removes all the maps but also all the annotations at the same time.
"Hide this one" hides the selected region(s) and all their components, leaving all the rest visible.
"Reveal all" restores the visibility of all regions and all their components. There are a couple of exceptions. First, if regions are in Solid display mode, they will be restored to wireframe display mode. Second, the usual procedure of automatically hiding regions in 2D mode that would otherwise block the view still applies. Finally, the status of the toggles in "Region mapping" (Analysis menu) is not affected.
Create region figures
This panel sets up the display for creating figures. In Blender, this is done by setting up a virtual camera, which has an adjustable field of view and distance rendering limits. These options are designed to simplify this for creating simple 2D figures. For animations, which require more advanced controls, see the "Camera animation control" panel below.
To create figures, there must be exactly one region selected. Like annotations, each virtual camera will be associated with its parent region. Figures can be created for as many regions as the user needs. By default, cameras are visible (as pyramidal structures, where the direction from the tip to the base of the pyramid indicates the viewing angle) in the 3D view, which can be messy and confusing. Their display can be toggled using the toggle button in the "Region mapping" panel of the Analysis tab.
Note that cameras created will only function correctly in the current viewing mode. If you switch from 3D to 2D, or vice-versa, you'll need to use "Clear camera" and then set up a new camera. As with other region-based operations in FRELLED, all the parameters which control the camera behaviour will only take effect when the associated region is selected.
"Clear camera" deletes the camera and its associated components for the currently selected region.
"Image edit" creates a camera with sensible default settings for the specified region, or switches to the existing camera if already present.
"FOV" defines the field of view in Blender units. "0" will set this automatically based on the size of the region.
"X/Ypix" sliders set the size of the image to render in pixels. Higher resolution images are generally of higher quality but of larger file size.
The "OSA" toggle button sets whether or not to use oversampling. If enabled, the rendered figure will use Blender's internal render engine. If the FITS file is visible, this can be EXTREMELY slow. Disabling OSA will use the realtime 3D view, which includes a cruder form of anti-aliasing, hence the resultant images may have jagged edges but will render MUCH faster.
"Save image" renders the desired figure and saves it in the "Figures" subdirectory. The file format is PNG, and the file name is based on the selected region.
If you create a figure which includes a region visible within it, the display of this region depends on the display mode of the region (e.g. wireframe or solid) and whether OSA (or equivalently "Full render") are active. With OSA active, both wireframe and solid regions will show with their specified colours, either as an outline or a filled box respectively. However if OSA is not active, the colour of wireframe regions will revert to the colour used for the axes, as set in the "Main display options" of the "Figures" tab. This is because Blender does not have the capability to show different wireframe colours for individual objects in realtime.
Solid regions are always shown as filled boxes of the colour specified in the "Region setup" panel, regardless of the OSA status.
Camera animation control
This panel is hidden by default as it contains somewhat more advanced options.
This panel lets you specify the camera movement for creating simple animations, as well as adjusting other parameters not possible in the simpler "Create region figures panel". For animating the parameters used in the display of the data itself, see the "Animate ALL THE THINGS !" panel.
Note that some operations here, especially changing the camera's distance or field of view, can occasionally cause its clipping parameter (which governs the maximum distance at which objects are visible) to be incorrect. This can result in part or even all of the data being hidden. Normally this fixes itself, but it can get stuck. The clipping parameter cannot be set directly in the GUI, but manually adjusting the distance parameter (see below) normally causes it to update to something sensible.
The "enable" button sets up a virtual camera for the selected region, or switches to the view of an existing camera if present. NOTE : All operations then require the parent region to be selected, NOT the camera itself ! As with the "Create region figures" panel, cameras created will only function correctly in the current viewing mode. If you switch from 3D to 2D, or vice-versa, you'll need to use "Clear camera" and then set up a new camera. As with other region-based operations in FRELLED, all the parameters which control the camera behaviour will only take effect when the associated region is selected.
"Cube/Region" sets whether the new camera should point at the selected region or instead target the centre of the whole data cube. But it also has important additional functionality ! "Cube" cameras will keep everything visible when saving the image : annotations, maps, and everything, from all regions - unless the user has hidden things themselves. "Region" cameras automatically disable the visibility of all components not associated with the currently-selected regions. Their visibility is restored after rendering, so the user won't notice anything in the realtime display, only in the saved images.
The file browser box sets the prefix to use when saving the rendered images in an animation. When animating, FRELLED outputs a sequence of PNG images. This means that if the process is interrupted it is relatively easy to resume, though you'll have to figure out your own method for converting an image sequence into a movie file.
"Camera rotation and target position" : this row sets the current camera altitude and azimuth angle (degrees) and the positional offset of the camera target from the centre of the region/cube. These parameters can be altered interactively once a camera is set up.
"Orthographic/perspective" toggle sets the viewing mode for the selected region's camera. Not that this also switches the camera's FOV units : pixels in orthographic, degrees in perspective mode (see below).
"Dist" : scale the camera's distance from the target. In orthographic mode this doesn't do much since the view is perspective invariant; in perspective mode this controls what's visible within the camera's FOV.
"FOV" : sets the camera's field of view; as above, pixels in orthographic mode and degrees in perspective mode. Note that by the nature of Blender's camera system, in perspective mode you should keep the FOV to less than 180 degrees : more than this will have the view upside-down, while a value equal to this will show nothing at all. No such restriction applies to orthographic mode.
"X/Ypix" : as in the "Create region figures" panel, these set the size of the image to render in pixels.
"OpenGL / Full render" : toggles between using Blender's internal render engine (slow, especially if the FITS file itself is visible, but high quality) and capturing the 3D window display directly (very fast, but not of quite such high quality). The OpenGL view is generally sufficient for most purposes.
"Frame control" : sets the animation parameters. The first box sets the start frame of the animation to begin rendering from, the second the frame to stop rendering on. The third sets how many frames the camera should take to execute a complete 360 degree rotation around its target, always starting from 1.
"Render camera animation" does exactly what it bloody well sounds like.
Animate ALL THE THINGS !
Another panel which must be hidden so new users don't accidentally stab themselves in the eye with it or something.
This panel lets you animate the display parameters of the FITS file itself, rather than moving the camera. Only parameters which are real, numerical values can be animated. Discrete, integer, and non-numerical values will not be animated, and FRELLED will use the current value for these throughout the animation, e.g. the colour scheme.
By default, all possible display parameters will be animated for a single specified FITS file. It is also possible to use a different FITS file per frame, for example in a time series from a simulation, using the "Animate FITS" toggle button. If enabled, FITS files must have the same prefix and counting syntax, e.g. MyFile_001.fits, MyFile_002.fits, etc.
When setting up animations using this panel, the camera animation is disabled by default, but can be enabled with the "Animate camera" checkbox.
The features of this panel can also be used via an external script - see the "Bonus Features" section.
To use this panel via the GUI :
- Navigate to the first frame of the animation, typically 1. Do this using the timeline slider at the bottom, either by left-click-drag to place the green line, using the keyboard cursor keys, or by setting the value directly in the "Current Frame" box next to the playback control buttons (play, fast forward etc.).
- Set the initial values of all the parameters you want to animate, for example, the data range to use or the opacity.
- Press the "Set start display" button.
- Navigate to the final frame of the animation, set the final values of the parameters to animate, press the "Set end display" button.
- Press the "Prepare animation" button. This will interpolate all the values between the initial and final values specified for the start and end frame, written as a series of text files stored in the "AnimationDisplayFiles" subdirectory. If using logarithmic values, this is accounted for.
- The animation will not start automatically. Instead, save the .blend file (use Blender's file menu, probably it's a good idea to save this as a new file !) then re-open the file. This will now begin the animation. Once complete, the animation will automatically be disabled and you can open the file as normal.
Some words of warning. Animating the data display means re-converting the file into PNGs for every single frame, meaning you can easily end up producing tens of thousands of files. Since these are overwritten each frame, they won't take up too much space, but this can be taxing on an SSD drive. It can also be a very slow process. Try doing some test animations first : for instance, use the correct initial and final values of opacity (say) but only 10 frames instead of a 100, rendered at a lower resolution.
If something goes wrong and you do need to interrupt the animation, you can resume later. Check the animation control file in the "AnimationDisplayFiles" subdirectory, which sets whether to do an animation or not and which display files to use.
FITS Developer tab
This menu provides tools mainly intended for debugging rather than analysis or visualisation. They are not as well-tested as the other operations and shouldn't really be used unless absolutely necessary and/or you know what you're doing. Nonetheless, they're documented here.
Auto view control
This panel contains buttons for controlling the different layers Blender displays.
Modal Ops : FRELLED runs a background script which continuously evaluates which layers to display. In Blender, scripts which run continuously are invoked by so-called "modal operators". This button shows and controls the status of the operator which runs the layer-displaying script. If deactivated, both the operator and its script are completely stopped. They can be reactivated through the toggle.
This button can be useful if you accidentally press the escape key. This also causes the script to stop, so you won't see all the projections of the data but only the one which was last visible. Simply deactivate this toggle button and re-enable it.
Auto views : Similar to the above, this sets whether the layer-displaying script actually does anything, without altering whether the modal operator is itself active. Both toggles must be activated for the layer-displaying script to run. While the first toggle can be more useful in guaranteeing that the script is definitely not running, which is necessary for some operations (see below), this second toggle is more useful as this parameter is more often called by the FRELLED scripts : if there is a problem, it's more likely to be here than with the modal operator.
Show/hide header : By default Blender's windows have a header bar (actually shown at the bottom of the screen) where the user can access tools to manipulate objects, set the shading mode, and control the layers. This is hidden in FRELLED in order to avoid scaring astronomers, with some of its functions absorbed into the FRELLED menus in a more user-friendly format. This buttons toggles the visibility of the header, which includes a layer manager that sets which layers are visible. Note that Blender is hardcoded to force at least one layer to be visible at all times - it is strictly impossible to disable the view of ALL layers.
Show all layers : If the "Auto views" toggle is disabled, this button will make all layers visible. This is a quick way to see everything in the file at once. Note that this may cause a significant slowdown - it is recommended to be in "plain surfaces" or "bounding box" display mode (see the "Advanced file options" panel) first ! Showing the layer manager, as above, is a good way to check if this button is working.
Show tabs : On the left side of the screen, by default FRELLED shows only its four unique tabs (Display, Analysis, Figures and Developer). All the usual Blender tabs which normally appear here are disabled. These can be restored and deactivated using these buttons. It may be necessary to ensure the modal operator is fully deactivated before using these.
Selection controls
FRELLED automatically makes some objects invisible and unselectable (both via Blender's GUI and using Python scripts). This prevents objects from being accidentally moved or deleted. The four buttons here allow the user to override this.
Reveal all : this reveals all hidden objects in the file. Use the "Auto view control" panel above to change which layers are visible, and/or Blender's built-in layer controls, to see what's on the each layer. Use Blender's scene selector, or FRELLED's 3D/2D mode selector, to check what's in each scene in the file. Be warned that this can cause the display to become quite unpleasant, since you'll now see the full set of 9,000 blank images planes that FRELLED comes pre-loaded with, i.e. long white cubes getting in the way of everything.
Restore visibilities : prior to setting their visibility/selectability status, both "Reveal all" and "Make all selectable" generate a list of the current visibility and selectability status for all objects in the file. This button restores the visibility status using that list.
Make all selectable : allows all objects in the file to be selectable. Does not by itself make those objects visible (use "Reveal all") or actually select them.
Restore selection : as with "Restore visibilities", this restores the selectability of the objects based on the existing list. Deselects objects which should no longer be selectable, if they are currently selected.
Below this, an additional interface allows the user to alter the settings for an individual object by name. Users can control whether the object is visible, selectable, actually selected, or the active object in the scene. Entering "None" in the text box will ensure there is no active/selected objects in the scene.
The "Delete" button here will delete all the currently selected objects, along with all their associated materials and textures - even if they are linked to multiple objects.
Contour controls
In this panel you can enable/disable the vertex limit, or alter its value. This controls how many vertices are allowed in contours and isosurfaces. If FRELLED tries to plot something exceeding this, the contour or surface will be skipped and it will attempt to plot the next level in the sequence. The reason for this is that vertices can be memory-intensive and take significantly longer to generate, though this is strongly system-dependent. More information is given in the help files.
Deletion controls
These buttons let you delete various components from the file. This is not usually a good idea and should only be used as a last resort ! Their main use is when some modification is made to the FRELLED files/code that requires altering the basic setup, which means the whole thing must be recreated. This is very slow and best avoided if at all possible, but is occasionally necessary.
Note that all of these operations only affect objects, materials and textures within the file. The screen layout and scripts (internal text files) are unaltered.
Three buttons let the user delete different objects in the file :
Remove FITS images : This deletes the plane objects which are used for displaying the FITS data. Ordinarily these make up the vast bulk of the file so this can be a very slow process (~30 minutes). Only the image planes (and their associated materials and textures) are deleted, everything else is unaffected. This includes other objects in the file which are necessary for basic functionality.
Remove other stuff : This deletes every object in the file which is NOT an image plane or other component necessary for FRELLED operation - e.g. all user-created regions, maps and cameras. This is somewhat more convenient than just setting up a new file if you're happy with the data display options, though you could just disable the view of the components if you don't like them. Useful for debugging if some object is behaving unexpectedly.
Kill 'em all : The nuclear option. Wipes out ALL objects, materials and textures in the file, including those necessary for basic functionality. This is not usually a good idea.
As well as the buttons, three toggles are included to help prevent users accidentally doing something they'd regret. The "FITS" toggle must be enabled to delete image planes. "Others" must be enabled to delete the other components. For deleting everything, all three toggles ("FITS", "Others" and "Nuclear") must be enabled.
Note that after running these scripts, the data may still be present within the .blend file even if it's not accessible through the outliner. To ensure everything is completely removed, save the file, close it and then re-open. It can be a good idea to do this several times, and fully close Blender rather than just saving and re-loading the file with Blender still open. Check the file size to be sure everything is really gone : the default FRELLED.blend file is ~61 MB in size.
While these buttons aren't something you should use regularly, there are legitimate uses for them that may be of use to specialist users, as described below.
Volumetric setup
These buttons let you replace essential components of the file after deleting them. If you've nuked your FRELLED.blend file, this panel should let you recreate everything you need.
The first row of buttons deals with creating the image planes (sometimes referred to as "blanks") which are used for displaying the slices of the data. You should have deleted these already before trying to replace them - they will not be automatically removed if present; ideally, save the file and close/reopen a few times to ensure everything is fully remove from memory.
Create blanks operates differently depending on the viewing mode. In the default 3D viewing mode, it will create the number of blanks as described below. In 2D mode it will ignore this number and create the single blank needed for each projection. Note that it ONLY operates on one mode at a time; in 3D mode it doesn't set up the 2D blanks, and vice-versa.
The default FRELLED file comes with 1,500 image planes along each projection, i.e. 9,000 total : two sets per projection, since two are needed to allow forward and reverse viewing angles. This number has been carefully found through trial and error to strike a balance between creation speed, data volume display capability, and performance.
In practise, the number of blanks corresponds to the maximum size of the data set FRELLED can display in terms of pixel size along its largest axis. If your cube has more than 1,500 channels or pixels along any side, the default behaviour is to delete the existing blanks and recreate them with the exact number required. This can be extremely slow (see below), so if you're regularly using larger files, it may be worthwhile to manually alter the master FRELLED.blend file to a better number. Similarly, if you're regularly using very small files, you might consider recreating only a smaller number of blanks, which can substantially improve performance, especially loading times.
Note that creation speed is NOT negligible, as Blender is not well-optimised for dealing with large numbers of objects. For this reason the default setting of the number slider is 500. The time requirement is non-linear. For comparision, on the development system, 500 blanks takes 2 minutes to create, whereas 1500 takes ~30 minutes. This is why the override toggle must be enabled if the value is set above 1,000, to prevent the user from accidentally getting stuck with a process that might take much longer than they expect.
The hard maximum value the slider can take is one million. This is absurdly higher than anything remotely feasible on the largest system, since this is largely a limitation of Blender itself rather than hardware. The largest file I've ever worked with had about 30,000 objects, and that was a painful hell to use.
The second section allows you to append other potentially missing objects from another .blend file. For example there is a provided file, "MasterObjects.blend", in the ExtraScripts subdirectory (though you can use any file you like). This contains only a few objects : empties, a camera, and in particular a mesh with a special modifier setup that's used for the SumFlux script (this enables it to create a filled grid in an irregular shape). Specify the file using the file browser, then press "Append" to load the objects.
The button attempts to import ALL objects within the target .blend file. However, it is designed specifically to import the premade objects (excluding image planes) which FRELLED requires. To that end, all objects with names beginning '3D' are imported to the scene called '3D', and similarly for '2D'. Additionally, objects which are named 'MasterMainEmpty' and 'SpecialCubes' are placed on the 3D scene.
Any objects in the target file with a naming conflict in the current file are not imported, and a warning is printed to the terminal.
Restore defaults
This section lets you restore the default values for all properties set up in the FRELLED tabs (menus). This includes ALL values set by the corresponding scripts, including those not visible in the GUI. Useful if you really want to be sure the file is in a pristine, theoretically-functional state.
The major use for this is debugging. All the menus set up their parameters by setting default values, however the default values are only used if the parameter does not already exist. If a parameter is set somewhere it will retain this value when the .blend file is saved, so if it is set improperly (for whatever reason) and not restored this can cause problems.
While the Display and Figures tab include the option to save to, and load from, all their parameters to text files (there is even a DefaultDisplay.txt file included in the main directory), the other scripts do not - many of their parameters are overridden anyway, depending on the specific files. However you can in effect set your own default values. Load the master FRELLED.blend file, set all parameters to what you want and save the .blend file.
Default parameters for all scripts can be found in the RestoreDefaultProperties.py script.
Bonus features
There are a few features of FRELLED which don't use the GUI inside Blender. Some of these features are semi-experimental, and generally these are not as well-tested or as frequently used as the GUI features. If things go wrong, do get in touch at feedback @ rhysy.net. Usually the problems are trivial to solve.
Animation by external script
It's possible to set up animations in much the same way as in the "Animate all the things !" panel without ever opening Blender, or at least never interacting with it. In the /FRELLED/FRELLEDScripts directory is a file "External_AnimationFiles.py". This is capable of generating all the display files and the animation control file needed for FRELLED to render an animation. The script should be self-explanatory, but there are many parameters you'll need to set : the display files require entries for ALL possible parameters to be set.
The parameters for the script are all set inside the script itself, so make a copy of this first. Then go through and set every parameter. For those which can be animated, use the format "[value for first frame, value for last frame]", setting these two values to be equal if you don't want them to be animated. Then run the script using Blender's Python. If you didn't set up an alias to this, the easiest way is to copy the script into the directory containing the Blender binary or executable file and then run (on Linux) :
./blender --background --python External_AnimationFiles.pyOr on Windows :
./blender.exe --background --python External_AnimationFiles.pyBe advised this will immediately start rendering the animation.
This feature should work, but watch out for the display files it produces. The structure of these is sometimes changed and I don't actively check that this script is still producing compatible files. However, the script is very simple. Have a look at the included "DefaultDisplayFile.txt" and ensure that the script is producing files of the same structure. FRELLED relies on display files having identical format and structure.
Data processing techniques
Sometimes the data needs a little massaging to get it to look nice for visual analysis. Remember, FRELLED can run its analysis operations on a different file to the one it uses for display, giving the best of both worlds : use the file processed for the benefit of the human eye to do the inspection, but the original file for quantitative analysis.
Two scripts are provided in the /FRELLED/ExtraScripts folder for this purpose (plus a third which doesn't change the data for visualisation except to make it compatible for loading into FRELLED). Both of these have parameters which must be set inside the scripts themselves. All three scripts should be robust and generally just work.
The "AGESCleaning.py" script was developed for processing data from the Arecibo Galaxy Environment Survey. It has a number of options, each of which can be toggled independently :
- Sigma-clipped polynomial spectral bandpass subtraction. Fits and subtracts a polynomial (default is second order) to the spectral axis, removing large-scale variations. The "sigma clipping" means that it iteratively ignores points more than 2 sigma above the median value of the baseline, which is a good way to ensure real galaxies are not included in the polynomial fit.
- Recast to signal-to-noise ratio. As above, but divides each spectrum by its sigma-clipped rms. This converts the entire cube to S/N values. There are several advantages to this : it makes it very much easier to estimate good data display values since these will no longer depend on observational parameters; where there are strong spatial variations in S/N (e.g. at the edges of the cube), this avoids having the noisier regions appear brighter which obstructs the view of more sensitive areas. Of course, this cube cannot be used for estimating true flux levels.
- Find the minimum (or median) of the medians of the spatial bandpass. A.k.a. "MINMED" and "MEDMED". These split the spatial bandpass into an arbitrary number (default is 5) boxes. The median value of the bandpass is calculated within each, then either the minimum or median of the medians is subtracted from the whole bandpass at that location (works along slices of constant y-axis, generally Declination). This can improve accuracy significantly when there are very bright, extended sources present.
- Hanning smoothing. Uses miriad to do ordinary, bog-standard Hanning smoothing, which is usually a good idea. Nothing fancy about this except that it also reorders the structure of the cube to make it faster (and reorders it back once the smoothing is complete).
The second script, "Normalise.py", was included directly in earlier versions of FRELLED, but I've found it easier to split this off. It's designed for cases where there is significant levels of emission present throughout a cube but which strongly varies in dynamic range. Specifically, in Galactic HI data, there are clouds of HI present in every direction, but the average brightness level varies strongly from channel to channel. This means the view in 3D is utterly dominated by a few channels, with the rest of the data being obscured.
The script proceeds to inspect each channel and scale them according to the maximum value present in each channel, so the data range is constrained to vary between 0 and 1 everywhere. As with recasting to S/N, this is very good for enhancing the visibility of structures, but has the significant penalty that the cube cannot then be used to analyse flux values.
A third script, FixAxes.py is for dealing with files which have a structure incompatible with FRELLED. FRELLED requires that FITS file contain a 3D array, whereas in some cases 4D arrays are used. This script reshapes a provided FITS file to the FRELLED-compatible format. See the script itself for instructions on its use.
Virtual reality
Two scripts are included inside the FRELLED.blend which provide experimental support to export the data into a format viewable with more modern versions of Blender, tested in 2.91 and 4.0. They have limited functionality but are extremely simple to use. Both are operated in the same way, though one is designed for isosurfaces and the other for volumetric data. The basic procedure is :
- Load your data, either volumetrically or as isosurfaces, in the usual way.
- Save your .blend file ! This is important as this file is going to be used as the source for the VR import.
- Access the code as described here and select either the "ConvertIsosToVR.py" or "ConvertVolumeToVR.py" script (do NOT use the versions with "_OLD" in their names !). The scripts are very short and you can probably just run them. If necessary you can edit them to change the name of the output file but that's about it. All they do is export the names of the objects and some material properties to a text file.
- From the "ExtraScripts" subdirectory, copy either of the files "VR_Isosurface_Import_2.91.blend" or "VR_Volume_Import_2.91.blend" into the main directory, where the FRELLED file you were just working with is located. Open this new file in Blender 2.91 or 4.0 (probably any version from 2.80 onwards will work).
- Each .blend file contains a single script. Scan through it and edit the name of the .blend file to import from, and if necessary also the text file. Then you can just run it and everything should work.
- Change your display to "rendered", connect your headset, press "Start VR session", and off you go !
(I'll add more details here if anyone wants any, at the moment I'm assuming VR users understand a bit more about Blender than typical FRELLED users. This will be made more user-friendly / foolproof in due course).
If you're using an Oculus Quest with Virtual Desktop (and possibly AirLink) for wireless PC access, be sure to start Steam VR from within Virtual Desktop first ! Once this is done, the "Start VR session" button within Blender should work as normal, but if you don't do this, Blender won't recognise the headset connection. Of course if you connect via cable this is not a problem.
Note you cannot do very much besides view the data. If you select the empty, you can grab, move, rescale and rotate the data. In Blender 4.0 you can grab and rescale the entire scene using the controllers.
Using Blender as a VR viewer is functional but nowhere near optimal. A much faster, far higher-performance VR viewer designed for FITS files is iDaVIE. The only real, though not insignificant, advantage of Blender is that iDaVIE does volumetric display and nothing much else, whereas Blender can also do heightfields, isosurfaces, annotations etc. But in terms of viewing volumetric data, iDaVIE wipes the floor with Blender and spits on what's left.
Under the bonnet
Blender itself is open source but I know absolutely nothing about accessing its source code. All of the FRELLED code is accessible directly within the FRELLED.blend file. This section gives a brief guide to understanding how to access the code and how it's organised. It is NOT intended as a line-by-line pseudocode or detailed instructions as to how to write Python code for Blender, though some basics are included.
How FRELLED displays files
At its core, FRELLED works by converting FITS files into sequences of PNG images, one sequence per projection (XY, XZ, and ZY). Each PNG image represents a slice of the file. For example, in each file the XY projection, which usually maps RA and Dec, would correspond to a single velocity channel. Depending on the settings the user chooses, the file could use different transfer functions for colour and transparency, with the latter encoded in the PNGs alpha channel. Within Blender, these PNGs are shown on transparent plane meshes, with a background script constantly evaluating which set of PNGs images to show for the optimum display. These PNGs are used for display purposes only : whenever FRELLED does any operation requiring the original FITS data (e.g. calculating flux) it uses this instead.
Meshes are a type of Blender object which are displayed as polygons defined by a series of vertices with Cartesian XYZ coordinates. Other objects in Blender, such as cameras, have different attributes. FRELLED uses a combination of the different objects for the user to display and interact with the FITS file. These are all organised within different scenes. Scenes are 3D workspaces which are themselves organised into 20 layers. A layer is a way of toggling the display of a set of objects. At least one layer is always visible (Blender is hardcoded to ensure this) but all 20 can be viewed simultaneously. Each layer shows the same 3D workspace, and can contain any number of objects. Objects can be shared across multiple layers, though not scenes.
The visibility of layers can be toggled graphically, see the Developer tab. This reveals the layer manager. Each box represent a layer. A darker shade indicates that layer is currently visible, while small dots within the layers indicate they contain at least one object. Note that objects can be shared across different layers.
FRELLED uses the layers to organise its objects as follows :
0 - 3D mode : XY projection, from front; 2D mode : XY image
1 - 3D mode : XZ projection, from behind; 2D mode : XZ image
2 - 3D mode : ZY projection, from right; 2D mode : ZY image
3 - Region axes
4 - Main axes and regions
5 - Integrated flux maps (moment 0); peak flux maps
6 - Integrated flux contours (moment 0); peak flux contours
7 - Renzograms
8 - Velocity maps (moment 1)
9 - Figure annotations
10 - 3D mode : XY projection, from behind
11 - 3D mode : XZ projection, from front
12 - 3D mode : ZY projection, from left
13 - Velocity dispersion map (moment 2)
14 - SDSS RGB images
15 - Isosurfaces
16 - Cameras
17 - Point clouds
18 - NOT USED
19 - Markers and a specially modified mesh object for creating point clouds
Normally, the FRELLED GUI should be used to toggle these, though you can override this with the Developer tab [TO BE WRITTEN]. FRELLED also contains several background scripts which are running continuously to set the most appropriate view, ensuring the demand on the GPU is minimised.
In addition to layers, objects themselves have a visibility attribute - they can be hidden from the real-time view, the render display, or both.
Blender is used for creating both still images and animations. To that end, the current frame is displayed as a numerical bracketed number in the lower-left of the 3D viewport, e.g. (1) by default.
2D mode is a bit different than 3D mode. In 3D, a whole series of meshes are used for the PNG sequence for each projection. In 2D mode, there is only one mesh per projection. Instead, the image that mesh displays changes as the frame is updated.
Accessing the code
Using the "Screen selector" drop-down menu, choose "Scripting", or better yet "Scripting fast" (see the Interface video at 1:36). These are both very similar, but the latter has a larger code window, and for some reason this tends to make Blender's GUI in this section far more responsive. God knows why. Anyway, this will show the following :
The 3D window is a reduced to a smaller section on the right. At the bottom is a Python interpreter (with the blue text) providing full access to Blender's Python distribution. The main part of the screen shows the Python scripts that constitute FRELLED. The "Script selector" drop-down menu allows the user to choose between different Python scripts.
Each of these sections can be maximised with Ctrl+UpArrow with the mouse in the appropriate section, and Ctrl+DownArrow to restore the display. Alternatively, for the main script editor, use View -> Toggle Fullscreen Area (same horizontal banner as the "Script selector" highlighted in the above image)
Ordinarily, scripts can be run with the "Run script" button (again in the same banner), or via the menu Text -> Run Python script, or by pressing Alt+P with the mouse in the script window. However, the structure of the code is organised such that it does not entirely run a series of scripts in linear fashion. Some are run constantly in the background : checking the orientation of the viewpoint so that the correct projection of the FITS data is displayed; updating every time the frame is changed for animations and heightfield displays.
Code for the GUI menus is executed in its entirety, and Blender tries to run as much as it can. Only a really serious error will prevent the menus from being displayed; sometimes only part of a menu will be shown if a problem is encountered. Problems with the non-GUI scripts will not affect the display of the menus at all.
FRELLED file structures
The FRELLED main package contains the following :
- FRELLED.blend
- M33.fits
- ReadMe.txt
- Changelog.txt
- DefaultDisplayFile.txt
- ExtraScripts :
-- Files.txt
-- FitsBaselineSN.py
-- FixAxes.py
-- fontlist-v300.json
-- fontlist-v310.json
-- MakeFRELLEDScript.py
-- Normalise.py
-- VR_Isosurface_Import_2.91.blend
-- VR_Volume_Import_2.91.blend
- FRELLEDScripts :
-- External_AnimationFiles.py
-- External_RenderImages.py
- HelpFiles :
-- Analysis
-- Figures
-- GUIAndGeneralThe Python scripts in these subdirectories can be edited as with any other Python script. They are called by FRELLED when certain actions are executed, and use Blender's Python but are run externally to Blender.
In more detail :
FRELLED.blend : the main interface to FRELLED.
M33.fits : an example FITS file provided for testing.
ReadMe.txt : a shorter guide to installing and running FRELLED.
Changelog.txt : lists updates to FRELLED from previous versions.
DefaultDisplayFile.txt : for setting the defaults used in the main FITS Display tab.
ExtraScripts : a directory containing files used for fixing installation problems, processing FITS files for better display, and for converting FRELLED files into a format for virtual reality display in Blender 2.91.
FRELLEDScripts : a directory containing files used for converting the FITS files into PNGs which it uses for display, and for producing display files used for animating data display and other values.
HelpFiles : a series of subdirectories containing simple text files explaining each panel. Note also "General Usage.txt" in "GUIAndGeneral".
In use, FRELLED creates additional files and directories as needed. The major ones are as follows :
XY
XZ
ZY
VelocityMaps
RotationMovie
Mbspect
Figures
SDSS
FitsHeader.txt
OperationsFile.txtWhich in more detail are :
XY/XZ/ZY : Directories containing the PNG sequences for each projection.
VelocityMaps : Directory containing PNG images to colour the file according to its spectral axis.
RotationMovie : Default directory to hold images when rendering turntable animations.
Mbspect : Directory containing input and measurement files when using mbspect to analyse spectra.
Figures : Directory for holding still figures.
SDSS : Directory holding SDSS RGB images for display within FRELLED.
FitsHeader.txt : A human-readable version of the FITS header of the file being shown.
OperationsFile.txt : A text file produced whenever it's easier to produce an external file than set internal variables used by FRELLED.
Description of the code
THIS SECTION INCOMPLETE. I'd prefer to eventually split this section off as its own page.
FRELLED contains around 60 internal scripts, which fall into a few different categories : 1) Setup; 2) Graphical menus; 3) Background functions; 4) Analysis and operations; 5) Developer tools; 6) Notes and examples. These are as follows.
Setup scripts
These scripts are run whenever the FRELLED.blend file is open. They configure the display and functionality of Blender to ensure all the other scripts work correctly.
MainSetup.py
This is the primary script that launches all the rest. This is run automatically on opening the FRELLED.blend file, hence the "Register" checkbox is selected (other scripts execute internal register functions - having them all registered at the start causes unpleasant conflicts). It runs the three GUI menu scripts (which provide the "FITS" tabs available on the left hand side of the screen), three background scripts (which update the view of the data depending on the current frame and viewing angle), and a special script to check whether animation mode should be used or not.
DisplaySetup.py
This script is not invoked directly from MainSetup.py but via the FRELLEDGUI.py script, which sets up the "FITS Display" GUI tab, and is itself called from MainSetup.py. The script optimises the Blender display preferences to be suitable for viewing FITS files, e.g. setting the background colour to black, setting the wireframe colour (used for the FITS axes) to white, and other settings needed to view the data correctly.
In general these changes occur whenever this script is run and do not alter the user's default settings for Blender. Closing Blender and re-opening a non-FRELLED file will retain the user's default colour schemes and other settings. There are two exceptions which do update the Blender preferences file :
- Multi-sampling is enabled. This is essentially anti-aliasing for Blender's real-time view, which is important for making nice presentable figures. The setting requires Blender be saved and re-opened, hence the preferences file must be updated. It should have minimal impact on Blender's performance and it general it's beneficial to keep it on. To deactivate it, do File -> User preferences -> System tab -> Choose "No MultiSample" from the drop-down menu in the middle section ("Window draw method").
- Enables registered Python scripts to run automatically. This is essential for FRELLED's menu's to appear and should not cause any issues elsewhere. To disable this, do File -> User preferences -> File -> Disable "Auto run Python scripts".
Graphical menus
Three scripts, "FRELLEDGUI.py", "FRELLEDAnalysis.py" and "FRELLEDFigures.py" respectively create the "FITS Display", "FITS Analysis" and "FITS Figures" tabs on the left-hand side of the screen. The operation of the various scripts is described above. In general, each button generated by the script calls another script to run the specified operation, except for very simple processes. This is probably not as strict as it should be, however, with some complex functions still included in the menu scripts themselves. This will eventually be replaced.
Background functions
These scripts are invoked by the MainSetup.py script. They are run constantly in the background, either whenever the mouse is moved or whenever the frame is changed. By necessity, such scripts need to be simple, and/or execute complex functions only whenever very specific conditions are met, in order to prevent a low frame rate in Blender's display.
ModalViews.py / 3DAutoView.py
An essential part of FRELLED's volume-display capability is that it does not show all the data at once. The user may import all three projections or they may need only one or two. These scripts, run continuously whenever the mouse is moved, evaluate which projection to display based on the orientation of the view. To simplify this, the "ModalViews.py" sets up the modal operator needed for this continuous evaluation - all this script really does is call "3DAutoView.py", which is where the actual view changing is done.
Each of the sequences of PNG images is imported onto plane objects which are held on different layers internally in Blender. The "3DAutoView.py" script selects which of these layers is the most appropriate to view, based on both the current orientation of the view, which projections have actually been imported, and which other objects should be visible (e.g. contours). For example if only the XY projection has been imported, then the user will only ever see layers 0 or 10, never those of the other projections, whereas if multiple projections have been imported, other layers may become visible.
Sometimes the view-switching can interfere with other operations. The other FRELLED scripts set a boolean variable to ensure this is disabled when necessary, e.g. moving objects between layers. The user can also press the ESC key to manually stop it, but this is a bad idea, so don't do that.
The 3DAutoView.py script has extra functionality in 2D mode. As well as setting which layers are visible, it also sets which image is displayed based on the currently visible projection and the frame (when the display of the cube has been stretched, FRELLED corrects for this). The position of the image mesh is set according to its position within the data, so that when the user creates region masks, they will correctly hide that part of the data. The script also updates the position of the 3D cursor, so that it is always at the position of the mesh as set along the axis perpendicular to the plane, e.g. :
SCREENSHOTS TO BE ADDED
AnimateMaskVisibility.py
This script only operates in 2D mode. In 3D mode, all regions are always visible. In 2D mode, some of them must be hidden otherwise they can block the view of data which should be visible. Whenever the frame is changed, this region check the current frame, the position of the image mesh, and sets the visibility attribute of each individual region depending on whether or not it extends over the currently-visible slice of the data or not.
AnimateHeightfield.py
This script only operates in 2D mode when heightfields are enabled. It checks if the heightfield mesh exists and has the correct number of vertices (but will not create it itself if a problem is encountered). If so, it displaces each vertices perpendicular to the plane by an amount corresponding to the flux of the data in the currently-visible slice.
Analysis and operations
The GUI and other scripts are designed to be modular, with complex or frequently-used subroutines placed in the own scripts. Some of these are called in their entirety whereas others are imported as functions.
AutoFindProjection.py
A sort of simplified version of "3DAutoView", this finds which projection should be displayed based on the current orientation of the view (though it does not account for which projections have actually been imported - it assumes all three have been loaded). This script is designed for and used by ScriptedAnimation.py, which renders animations according to sequences of external files.
CameraControl.py
This script creates and adjusts the cameras needed for rendering figures and animations. It can 1) create new cameras; 2) update existing cameras; 3) set the GUI values to those of an existing camera when selecting it. This is used in the "FRELLEDFigures.py" script. Cameras are oriented using empties, a special type of Blender object. Empties are not meshes : they contain a single value of position, rotation and scaling but nothing else. The cameras are parented to the empties, meaning that updating the empty's values affects the camera. This provides and easy way of setting up turntable-style rotation animations.
SCREENSHOT TO BE ADDED
CheckBlanks.py
The PNG images FRELLED loads are each displayed on plane meshes. Each plane mesh is named according to the slice of the data it is set to display. This script checks that every required plane mesh exists (this is not quite the same as simply counting the meshes, as the name of each mesh object is important). It sets a Blender-internal variable to False if any are missing, at which point it complains loudly to the user and falls over.
CheckCamera.py
Simply checks whether the selected region already has an associated camera and the required empties. Not sure why this is done separately to CameraControl.py, which includes much the same functionality.
CheckFITSHeader.py
Does a very simple check on the FITSHeader.txt file to see if the FITS file has a valid WCS or not, setting an internal parameter to True or False depending on the result. This reproduces some functionality contained in the WriteHeader.py script, but since it doesn't actually access the FITS file itself, this can be a lot faster.
CheckRegionName.py
Checks that the region name specified in the Analysis GUI menu does not contain characters (or sequences of characters) that might cause conflicts within FRELLED. For example, the naming syntax is generally "RegionName_00X", and FRELLED uses the underscore to find the numeric suffix, so using more underscores would screw everything up.
ConvertIsosToVR.py
Special script not called by any menus, used for converting isosurfaces for virtual reality display in Blender 2.91. See section LINK.
ConvertToVR.py
Special script not called by any menus, used for converting volumetric data for virtual reality display in Blender 2.91. See section LINK.
CreateBlanks.py
Script for creating the plane meshes used to display the FITS data. Because Blender is not optimised for even modest (>1,000) numbers of objects, this can be slow, so FRELLED comes with 1500 meshes per projection pre-loaded. This script is run if CheckBlanks.py reports false. For sake of speed, it will then create exactly the number of required meshes for the current file. Ordinarily, if the data set loaded is smaller than the 1500 slices available from the pre-computed meshes, the meshes which are not needed are hidden from view.
FakeAxes.py
Draws the axes of the cube in 3D and 2D mode, when no world coordinate system is available or the user explicitly enables this. Exactly the same as RealAxes.py except that it uses pixel values for the axes labels instead of world coordinates. This is a useful fallback since the number of possible coordinate systems is vast and it's difficult to account for all of them. Similarly, RealAxes.py is designed for sexigesimal coordinates, which are not appropriate for cubes spanning a very small angular range.
FixObjectSize.py
Ensure that a supplied object has only positive scaling values. Sometimes object scale, rather than true dimensions, are more convenient, but it is possible to set these to negative values using mouse gestures. This script prevents that so that their scale and dimensions will properly correspond.
GenerateIsos.py
Generates isosurfaces for the selected region(s) at the specified flux levels.
GetCut.py
Find the flux values corresponding to the specified percentage levels.
GetObjectCoords.py
Returns the location, scale and dimensions of a supplied object.
GetRegList.py
Goes through all the objects and adds them to a list if they are both selected and a region. Includes a workaround because the built-in Blender Python operator for accessing all selected objects does not work reliably. Also includes a routine for getting a list of unselected regions, which excludes the Solid regions as these just cause problems. Called as a function.
GetRegVis.py
Used for switching back and forth between 3D and 2D mode. Regions can be displayed as wireframes, solid, or in hybrid mode. This script gets their current status so that when we switch from 2D mode (where some regions are necessarily completely hidden) we restore their original status. Since we can't set lists as variables stored within the scene that can be accessed by other scripts, the script stores the list as a scene property in the form of a string, which can be re-converted to a list when required (in AnimateMaskVisibility.py).
GetStretch.py
For data already loaded into FRELLED, determines if the user decided to stretch the data display or not - compares the shape of the outline object with the information in the FITSHeader.txt file. Designed to be imported as a function which requires the "request" keyword. If set to "onlystretch", it returns only the factor by which the data is stretched. If set to "allcubesize", it returns the pixel sizes of the cube along each axis as well, in the following format : X size, Y size, Z size, stretch factor.
GetViewMode.py
Determines whether we're currently using orthographic, perspective, or camera mode in the 3D viewport.
HideAndReveal.py
Script that can hide or reveal either the selected regions and all their components, or the unselected regions (and all their components) provided at least one region is selected.
ImportRegions.py
Imports regions from a text file. Uses Blender coordinates if available as these also include scaling. Only uses world coordinates if Blender coordinates unavailable.
LoadImages.py
Loads the generated PNG images into FRELLED. This also sets the specified material parameters and the visibility of the plane meshes, so that the meshes which remain blank are hidden from view.
LoadImages2D.py
As for "LoadImages.py" except this operates only in 2D mode (somewhat simpler as there are no visibility requirements and less material settings to alter).
MakeMap_File.py
Includes three functions for basic image processing :
- makecustomcolours : Generates custom colour transfer functions as an addition to those available from matplotlib.
- makecolourimage : Converts a supplied image array into a 2D colour transfer function.
- getcolour : Takes a single fractional value and converts it to RGB values for all the available colour schemes.
MakeMap_Flux.py
Generates an integrated flux map (moment 0) from the selected region(s) as a PNG image, which is loaded (in the "FITSAnalysis.py" script) as an image plane in the same way as the main FITS data.
MakeMap_FluxContours.py
Creates contours as mesh objects based on the integrated flux (moment 0) of the selected regions.
MakeMap_Renzo.py
Creates contours as mesh objects based on a single specified flux value, for the selected regions.
MakeMap_SDSS.py
Downloads an RGB image from the SDSS corresponding to the field of view of the selected region(s). Saved as an image file and loaded onto a plane mesh in the FITSAnalysis.py script.
MakeMap_SumFlux.py
Calculate the sum of the flux values within the specified region assuming the region is a perfect and non-rotated cuboid. Fast and accurate. Returns raw and beam-corrected values.
MakeMap_SumFlux4.py
Ingeniously clever script to calculate the sum of the flux values within volumes of an arbitrary shape. First it applies a special modifier to the target region to convert it to a point cloud, with each vertex at integer position values in the original data (account for the display stretching, if any). Then it fills in the interior of the object, though this procedure may not be perfect. It then uses this final set of vertex positions to compute the total flux, returning the raw and beam-corrected values.
MakeMap_SumFlux5.py
Computes the sum of flux values for an existing point cloud. This allows the user to create the cloud as in SumFlux4 and then modify it manually as needed.
MakeMap_Transparency.py
Enables transparency for the selected map object, if necessary.
MakeMap_UpdateThickness.py
Updates thickness of the selected existing contours.
MakeMap_Velocity.py
Creates a velocity (moment 1) map of the selected regions(s), saved as a PNG file and loaded onto a plane mesh in the FITSAnalysis.py script.
RealAxes.py
Creates axes using world coordinates, either for the cube or selected region.
ScriptedAnimation.py
Executed from MainSetup.py, this checks the external AnimationControl.txt file. This file sets whether to try and render an animation or not. If disabled, this file does nothing. If enabled, it sequentially loads and applies the settings from each DisplayFile, rendering and saving an image for each.
SetTextureModes.py
Sets how the textures (PNG images from the FITS file) affect the material settings of the image planes.
TakeMeThere.py
Places the 3D cursor at the pixel position of the specified world coordinates, accounting for whether the FITS display has been stretched or not.
ViewActiveCam.py
Changes the viewpoint in the 3D viewport to the active camera, if one is present and enabled.
WhereAmI.py
Retrieves the world coordinates of the 3D cursor's current pixel position, converting into sexigesimal format.
WriteHeader.py
Writes the essential parameters of the FITS header for the chosen file (in the Display tab) to an external, human-readable file "FitsHeader.txt". Based on the parameters, this also sets an internal variable to check whether world coordinates can be used or not.
Developer tools
CreateBlanks_basic.py
Like CreateBlanks.py but can be run by the user, allowing them to specify how many meshes they want rather than having this based on the size of a FITS file. Useful when setting up FRELLED for general use.
EnableSelection
Many objects in FRELLED are invisible and/or unselectable in the GUI. This script can alter these statuses, which can be useful for debugging purposes.
KillEmAll
Unlike Vladimir Putin this one definitely isn't bluffing ! This will wipe out absolutely everything in a scene (maybe even the whole file) - textures, materials, and objects. However, it is not quite a "scorched Earth" policy. The window layout, scenes themselves, and the script files all remain. This essentially restores the file to a pristine state, from which the scripts can be used to recreate FRELLED in its blank but usable condition. Use this only if absolutely desperate.
This can take some time so be patient. For very large scenes you may want to modify this to do each section one at a time to avoid memory issues. Note that items may still remain in Blender's memory - to ensure they are definitely removed from the file, save it, close it, and re-open. You may need to do this several times : check the file size to see if you've really killed everything off or not.
RemoveBlanks.py
A much less drastic version of KillEmAll, this script removes all the image planes (despite the name, this includes planes which have textures loaded). Must be executed manually. As with KillEmAll, the file should be saved, closed, and re-opened to properly update it.
Notes and examples
EssentialNotes
Provides some instructions on the idiosyncrises of Blender, wherein some functions do not behave either as expected or advertised, and the need for workarounds. Also provides some useful Python commands for debugging.
HeightfieldTest.py
Simple example of creating a heightfield mesh.
Import2D_Example
Minimum working example of how to create meshes for the 2D display and load the FITS data.
MakeMap_SumFluxExp1.py
A couple of experimental alternatives for calculating the flux in arbitrary volumes. Firstly by attempting to compute (by various methods) whether a point is inside a mesh or not, and secondly by using Blender's boolean object operators. Neither work very well, even though they really should do in principle.
This script is heavily commented, so employing the different methods requires careful reading to un-commented the appropriate sections.
MakeMap_SumFluxExp2.py
Another failed experimental attempt to compute the sum of values within an arbitrary volume, this time by using the boolean operators on a regular grid mesh.
PopUpError.py
Illustrates how to raise a graphical pop-up notification with a specified message. Not sure this is actually used, though it probably should be.
PopUpExample
Shows how to raise a pop-up presenting the user with a choice. However, this turns out to be useless. Blender's menu code system is downright weird, and will not wait for the user to make the choice - it will execute the code regardless. There may be workarounds for this but they sound complicated.
Roadmap
FRELLED version 5 runs on Blender 2.79, a major upgrade from FRELLED versions <= 4 which were based in Blender 2.49. But Blender has a whole team of developers behind it and at the time of writing has moved on to Blender 4.0 ! Keeping pace with this is just not feasible for a single developer. Nevertheless, FRELLED 5 has been written to ensure that both maintaining (fixing bugs, adding what new features are still possible) and migrating the code to newer versions of Blender should be as simple as possible. This would not be a minor undertaking, but it shouldn't be anything like the scale of the rewrite that version 5 has been. The code syntax has changed between Blender 2.79 and later versions, but to a far, far smaller degree than the change from 2.49 to 2.50.
Improvements planned for the 5.X series
No doubt a host of undetected bugs still remain. Besides fixing these, there are many possible improvements that would be relatively minor to implement. These may be included in FRELLED 5, while more substantial changes I plan to leave until FRELLED 6. There doesn't seem much point in trying to make really major changes given that Blender 2.79 is already obsolete. In any case, such changes include :
- Quick Preview in 2D mode should only export a single image
- Ensure logarithmic scaling works in a consistent way in all cases, i.e. when dealing with negative values (I'm not sure this is currently the case)
- Ensure consistent way of deleting items (some scripts still use a crude method of renaming and unlinking objects, rather than the much more powerful delete operator)
- Isosurfaces should have a check for generating too many levels (too many objects is bad, even if they have no vertices) and/or vertices to prevent exceeding memory limits
- Should have colour bars plotted for all appropriate maps, not just renzograms
- Screen colours should refresh correctly on loading
- Queries which generate HTML files should be sorted by region name
- Miriad should allow user to specify a symmetrical x-range
- Mbspect should create some marker at the position-fitted coordinates, so the user can compare their initial guess with where mbspect thinks the source is
- Camera clipping distance should be included as a GUI parameter
- Allow queries to other services which can be accessed with only the sky coordinates, such as ESA Sky and the Legacy Survey
Towards FRELLED 6
Many other features are possible using modern versions of Blender which are simply impossible to do with Blender 2.79, including:
- Interactive colour transfer functions using EEVEE. I've tested this and it works. Of course, this would greatly reduce the need to reload the data with different clipping ranges, which would make life a lot easier !
- Native support for virtual reality. Not sure if this is actually of any scientific use but it's great for outreach. If Blender would implement some way of using the menus and controls within VR, then in fact I think iDaVIE does demonstrate, quite effectively, that there's a case to be made for scientific use here too.
In addition there are other features which could be implemented in FRELLED 5 but would be much more substantial than the ones in the section above. These I plan to leave until FRELLED 6 because major upgrades like these ought to be done when recoding the whole thing. They include :
- Integrate astroquery, use for automatically labelling known sources / objects in a field (might be doable to include this in FRELLED 5, we'll see)
- Support truly 2D FITS files, including heightfields. Seems a shame not to allow these, even if FRELLED is very much geared towards 3D data
- Totally robust coordinate systems to any scales, allow frequency and wavelength for plotting. A lot of work was done to FRELLED 5 to make this more robust to data on larger and smaller spatial scales than was possible in FRELLED <=4, as well as allowing a proper calculation of the spectral axis. Nonetheless, it's still not perfectly foolproof
- Restore n-body particles and vector plotting. These are the only major features from FRELLED <=4 which were not included in the rewrite as this would be a step too far. In modern Blender it's possible to have multi-coloured particles in the realtime display so this would be extremely useful for visualising simulations
- Restore old method of multi-volume rendering (independent blanks). This uses more memory but gives a much better, more controllable display with an interface that's easier to understand
- Support 4-axis cubes rather than insisting the user reshape them
- Help files should open in tkinter windows or something, rather than the system default text viewer
- Include the mbspect source code directly within FRELLED rather than calling miriad, or provide some equivalent spectral line analysis tool
The major differences in Blender 2.79 and later versions, from a FRELLED perspective, are :
- Layers are replaced by collections. These are much more powerful, and unlike layers, can be unlimited in number, allowing more components of the data to be shown (e.g. more moment map types).
- The material system must now use Cycles or EEVEE, both of which are extremely similar. The old materials cannot be converted on-the-fly. However, the VR conversion scripts in FRELLED 5 already demonstrate how to set up replacement materials.
- Selection state of objects is now handled by functions rather than object properties.
While there are no doubt many other more minor differences as well, hopefully, I managed to get one of FRELLED's menus working within Blender 2.8X without much difficulty. Since the code is now so much more modular, in principle things could be done on a script-by-script basis. So while FRELLED 6 is not happening any time soon, it's at least something on the horizon.
