[1]:

# =============================================================================
# PATH APPEND
# =============================================================================
import os, sys
sys.path.append(os.path.abspath('./voidFinderProject'))

[2]:

# =============================================================================
# IMPORTS
# =============================================================================
import os
import pathlib

import numpy as np

[3]:

# =============================================================================
# PATHS
# =============================================================================

path_tng = pathlib.Path(os.path.abspath("./datasets/halos_ascii_1000_1024_npmin_10_z0.51.dat")) # Path to the dataset
path_workdir = pathlib.Path(os.path.abspath("./workdir_analysis_2")) # Path to the workdir

path_zobov_outputs = 'tmpfbdpzqgg2025-04-22T20:23:28.576077+00:00'
path_svf_outputs = "tmp4yecx4pi2025-04-22T20:43:27.232553+00:00"

Bulding input object Box

Build the Box object importing the module io.read_table, this will allow for ascii input read (or any other pandas available format). Specify the column variables using names.

[4]:

# =============================================================================
# BOX
# =============================================================================
from voidfindertk.io import read_table

box = read_table(path_tng, names=['m','x','y','z','vx','vy','vz'])

[ ]:

# We can build a box with any of the later attributes:

box = read_table(path_tng, names=['x','y','z'])

Building the Void Finder model

Import the desired available void finders, and set the variables for their use. You can specify a directory that holds the directory with the output files in workdir.

Perform the search over the box using the method find:

[7]:

# =============================================================================
# BUILD MODELS
# =============================================================================
from voidfindertk import zobov, popcorn, svf_popcorn

zbv = zobov.ZobovVF(density_threshold=0.2, workdir=path_workdir, box_size=box.size())
spherical = svf_popcorn.SVFPopCorn(densth=-0.8, cores=16, workdir=path_workdir)
pop = popcorn.PopCorn(densth=-0.8, workdir=path_workdir, shot_noise_threshold=16)

[6]:

# =============================================================================
# PERFORM SEARCH
# =============================================================================

voids_zbv = zbv.find(box=box)
voids_spherical = spherical.find(box=box)
voids_pop = pop.find(box=box)

Persist and recover data from a past search

You can persist your void object using picke. Otherwise you can recover your past search with the method build_voids. This method recieves a dict object with instructions to perform your build, tipically options are:

run_work_dir : Path to the directory that holds the files of the past search.
box : The input catalogue

The PopCorn method has also an extra input:

build_popcorn : If True, performs the build step from popcorn.

The following functions use this to simplify the object re-build:

[8]:

# Import the Void Class
from voidfindertk.core import Voids

# Helper to read svf_popcorn and zobov files:

def reader_helper(model, box, dir):
    # Set the model_find_parameters.
    model_find_parameters = {
        "run_work_dir": model.workdir / dir,
        "box":box
    }
    # Call the build_voids method from the model.
    tracers, centers, extra = model.build_voids(model_find_parameters=model_find_parameters)
    # Use the outputs to build the final Void object.
    return Voids(
        method=model.__class__.__name__,
        box=box,
        tracers_in_voids_=tracers,
        centers_=centers,
        extra_=extra,
        )

def reader_helper_popcorn(model, box, dir, build=True):
    # Set the model_find_parameters.
    model_find_parameters = {
        "run_work_dir": model.workdir / dir,
        "box": box,
        "build_popcorn":build
    }
    # Call the build_voids method from the model.
    tracers, centers, extra =  model.build_voids(model_find_parameters=model_find_parameters)
    # Use the outputs to build the final Void object.
    return Voids(
        method=model.__class__.__name__,
        box=box,
        tracers_in_voids_=tracers,
        centers_=centers,
        extra_=extra,
        )

# Recover the objects from the files:

svf_voids = reader_helper(model=spherical, box=box, dir=path_svf_outputs)
zobov_voids = reader_helper(model=zbv, box=box, dir=path_zobov_outputs)
popcorn_voids = reader_helper_popcorn(model=pop, box=box, dir=path_svf_outputs, build=False)

Datos guardados en /home/jorgefederico/updates/vftk_1109/vftk_updated/test_update_rep/voidFinderProject/workdir_analysis_2/tmpfbdpzqgg2025-04-22T20:23:28.576077+00:00/part_vs_zone_ascii.txt
Datos guardados correctamente en /home/jorgefederico/updates/vftk_1109/vftk_updated/test_update_rep/voidFinderProject/workdir_analysis_2/tmpfbdpzqgg2025-04-22T20:23:28.576077+00:00/zones_vs_voids_ascii.txt

Model exploration

Models have information about the amunt of voids found.
You can access original inputs from the model from the extra_ attribute of the object
If the Void Catalogue has additional properties you will find them in Void.extra_.properties , or else, as pandas.DataFrame.
POPCORN voids have 2 set of properties correlated by index: spheres, and voids
Acces the center of the void as an array from Void.centers.
Acces the tracer list (index based) from Void.tracers

[11]:

# Retrieve properties of ZOBOV
zobov_voids.extra_.properties.head()

[11]:

	Void#	FileVoid#	CoreParticle	CoreDens	ZoneVol	Zone#Part	Void#Zones	VoidVol	Void#Part	VoidDensContrast	VoidProb	Tracers_in_void
0	1	29182	5853504	0.024460	2899.596	2071	25	11172.46	10015	8.211967	1.510000e-104	[5720094, 5526898, 5530280, 5535112, 5535116, ...
1	2	27800	5504005	0.027375	1827.330	1160	10595	3881550.00	3416629	7.305856	5.020000e-75	[78656, 10479, 10480, 10481, 10484, 14586, 145...
2	3	11721	2419068	0.031486	3843.677	2839	310	116495.50	99375	5.896360	2.680000e-41	[521204, 464275, 477191, 478624, 493395, 50674...
3	4	16154	3280825	0.030940	1422.366	712	1334	542419.70	451554	5.842445	2.850000e-40	[107890, 17446, 17447, 17449, 25171, 25172, 25...
4	5	32370	6546154	0.032252	1176.252	673	190	75163.24	63200	5.701917	1.110000e-37	[1804782, 1775636, 1775637, 1800244, 1775635, ...

[14]:

# Spheres from popcorn
popcorn_voids.extra_.spheres.head()

[14]:

	x	y	z	radius	fracvol	level	id
4	379.134888	7.765100	5.365120	17.627710	22944.390625	0.0	1
3	300.014099	25.474621	1.051236	22.598261	48340.769531	0.0	2
14	209.403107	15.710010	9.910415	16.590490	74667.007812	1.0	3
13	221.090897	36.862228	5.085893	24.643360	62688.570312	0.0	3
1	799.258118	34.712769	0.503005	18.398979	26089.759766	0.0	4

[15]:

# POPCORN voids
popcorn_voids.extra_.voids.head()

[15]:

	id	n_mem	volume	n_part	flag	tracers
4	1	1	22944.390625	30	-1.0	[119228, 131991, 140803, 180917, 191170, 19136...
3	2	1	48340.769531	64	-1.0	[134528, 148114, 159765, 162005, 169394, 21859...
13	3	2	74667.007812	101	-1.0	[70700, 70701, 70702, 74214, 116828, 126803, 1...
1	4	1	26089.759766	34	-1.0	[3491296, 3523914, 3554257, 3591532, 3633528, ...
0	5	1	16720.619141	21	-1.0	[3494751, 3496171, 3496172, 3513013, 3515294, ...

Perform Statistics with the found voids

Compute the Void Galaxy Cross Correlation Function (VGCF) and the Void Size Function (VSF) for our void samples.

Compute the bare VGCF using: Void.void_galaxy_corr specifying a max radius for the search.
Compute the bare VSF using: Void.void_size_function. This method requires a radius value, if the catalogue doesent have one it can be calculated using the Void.effective_radius for simple radius finding or the find_radius_and_clean for a more sophisticated search.
Void.effective_radius method, returns a value for each void in the catalogue, if it could not find a subdense region in the location of a void it will return nan, so it is advised to first clean this values before making statistical calculations.

[8]:

import numpy as np

# Method with native radius
rad = svf_voids.extra_.radius

# Methods wihout native radius:
rad_z = zobov_voids.effective_radius(method='density')
rad_pop = popcorn_voids.effective_radius(method='density')

# Clean the nan values
rad_z_ = rad_z[np.where(rad_z>0)[0]]
rad_pop_ = rad_pop[np.where(rad_pop>0)[0]]

/home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/voidfindertk/core/radius_finder.py:226: RuntimeWarning: divide by zero encountered in divide
  density_n_nat_d = (3 * n_nat) / (4 * np.pi * distance**3)

Plots for VGCF and VSF

Plot accessors exist for this methods, they can be used by: Voids.plot.void_size_function for the VSF and Voids.plot.void_galaxy_corr for the VGCF.

As inputs for this methods provide two:
- Keywords for the method itself
- Keywords for the plot

[15]:

ax1 = svf_voids.plot.void_size_function(radius=rad, n_step1=3, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3, c='red', label='SVF')

ax2 = zobov_voids.plot.void_size_function(radius=rad_z_, n_step1=3, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3,c='blue', label='ZOBOV')

ax3 = popcorn_voids.plot.void_size_function(radius=rad_pop_, n_step1=3, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3, c='green', label='POPCORN')

../_images/notebooks_application_tutorial_18_0.png

[16]:

ax = zobov_voids.plot.void_galaxy_corr(max_rad=30, label='ZOBOV')
ax = svf_voids.plot.void_galaxy_corr(max_rad=30, c='red', label='SVF')
ax = popcorn_voids.plot.void_galaxy_corr(max_rad=30, c='green', label='POPCORN')

../_images/notebooks_application_tutorial_19_0.png

Finding radius and cleaning in a sophisticated way.

The module Void.find_radius_and_clean allows both radius finding and cleaning the catalogue of voids. The output of this method will be a new Void with new parameteres.

Available options for cleaner_method are:
- overlap : Discards voids with lesser radius when overlapping
- cbl : Sophisticated cleaner allows for multiple cleaner options
Available options for radius_method are:
- extra : Brings the native radius of the catalogue if it has it.
- density : Perform radius search based on local density.
- volume : Maps Void volume to radius if volume is available in extra. You can serch this property in the property catalogue of the method and add to extra.

[9]:

%%bash
module load gcc gsl boost cfitsio fftw

[10]:

new_svf_voids = svf_voids.find_radius_and_clean(cleaner_method='cbl', radius_method='extra', rescale=True)
new_zobov_voids = zobov_voids.find_radius_and_clean(cleaner_method='cbl', radius_method='density', initial_radius=False, rescale=True)
new_popcorn_voids = popcorn_voids.find_radius_and_clean(cleaner_method='cbl', radius_method='density', initial_radius=False, rescale=True)

CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpqzu8ml4mcbl_cleaner/input_xyz_rad.txt
CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpqzu8ml4mcbl_cleaner/input_tracers.txt

CBL > > > > > > >>>>>>>>>> CLEANING PROCEDURE STARTED  <<<<<<<<< < < < < <

CBL > Voids in the initial Catalogue: 5225

CBL >  --- Removing spurious voids --- 

CBL > Removed voids:
         r_min-r_max criterion: 0
         High central density: 801
CBL > Voids in the Catalogue: 4424
CBL > Time spent by the spurious voids-checking procedure: 0.901 seconds

CBL >  --- Rescaling radii ---

CBL > Removed voids:
         Bad rescaled: 765
         Out of range [10.00,100.00]: 1
         Cloud-in-void: 0
         Empty voids: 0
CBL > Time spent by the rescaling procedure: 16.507 seconds
CBL > Voids in the Catalogue: 3658

CBL >  --- Checking for overlapping voids ---

CBL > Catalogue ordered according to the central density

CBL > * * * Generating ChainMesh for void centres * * *

CBL > Voids removed to avoid overlap: 34
CBL > Time spent by the overlap-checking procedure: 0.034 seconds

CBL > Voids in the final Catalogue: 3624

CBL > Total time spent: 17.834 seconds

CBL > Writing the file: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpqzu8ml4mcbl_cleaner/cleaned_catalogue.txt...

/home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/voidfindertk/core/radius_finder.py:226: RuntimeWarning: divide by zero encountered in divide
  density_n_nat_d = (3 * n_nat) / (4 * np.pi * distance**3)

CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpu03dhenccbl_cleaner/input_xyz_rad.txt
CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpu03dhenccbl_cleaner/input_tracers.txt

CBL > > > > > > >>>>>>>>>> CLEANING PROCEDURE STARTED  <<<<<<<<< < < < < <

CBL > Voids in the initial Catalogue: 7192

CBL >  --- Removing spurious voids --- 

CBL > Removed voids:
         High central density: 2678
CBL > Voids in the Catalogue: 4514
CBL > Time spent by the spurious voids-checking procedure: 1.356 seconds

CBL >  --- Rescaling radii ---

CBL > Removed voids:
         Bad rescaled: 570
         Out of range [10.00,100.00]: 23
         Cloud-in-void: 5
         Empty voids: 5
CBL > Time spent by the rescaling procedure: 3.596 seconds
CBL > Voids in the Catalogue: 3916

CBL >  --- Checking for overlapping voids ---

CBL > Catalogue ordered according to the central density

CBL > * * * Generating ChainMesh for void centres * * *

CBL > Voids removed to avoid overlap: 242
CBL > Time spent by the overlap-checking procedure: 0.033 seconds

CBL > Voids in the final Catalogue: 3674

CBL > Total time spent: 5.539 seconds

CBL > Writing the file: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpu03dhenccbl_cleaner/cleaned_catalogue.txt...
CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpaeh4u9zgcbl_cleaner/input_xyz_rad.txt
CBL > Reading the catalogue: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpaeh4u9zgcbl_cleaner/input_tracers.txt

CBL > > > > > > >>>>>>>>>> CLEANING PROCEDURE STARTED  <<<<<<<<< < < < < <

CBL > Voids in the initial Catalogue: 3457

CBL >  --- Removing spurious voids --- 

CBL > Removed voids:
         High central density: 116
CBL > Voids in the Catalogue: 3341
CBL > Time spent by the spurious voids-checking procedure: 0.885 seconds

CBL >  --- Rescaling radii ---

CBL > Removed voids:
         Bad rescaled: 740
         Out of range [10.00,100.00]: 0
         Cloud-in-void: 0
         Empty voids: 0
CBL > Time spent by the rescaling procedure: 3.262 seconds
CBL > Voids in the Catalogue: 2601

CBL >  --- Checking for overlapping voids ---

CBL > Catalogue ordered according to the central density

CBL > * * * Generating ChainMesh for void centres * * *

CBL > Voids removed to avoid overlap: 14
CBL > Time spent by the overlap-checking procedure: 0.018 seconds

CBL > Voids in the final Catalogue: 2587

CBL > Total time spent: 4.527 seconds

CBL > Writing the file: /home/jorgefederico/updates/vftk_1109/vftk_updated/voidFinderProject/tmpaeh4u9zgcbl_cleaner/cleaned_catalogue.txt...

Now each new Void has the attribute radius:

[11]:

rad_svf = new_svf_voids.extra_.radius
rad_zobov = new_zobov_voids.extra_.radius
rad_popcorn = new_popcorn_voids.extra_.radius

[13]:

new_popcorn_voids.plot.void_size_function(radius=rad_svf, label='SBV', c='red' ,n_step1=1, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3)
new_zobov_voids.plot.void_size_function(radius=rad_zobov, label='ZOBOV', c='blue' ,n_step1=1, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3)
new_popcorn_voids.plot.void_size_function(radius=rad_popcorn, label='POPCORN', c='green' ,n_step1=1, n_step2=3, scale_1_num_samples=5, scale_2_num_samples=3)

[13]:

<Axes: title={'center': 'Void Size Function\n Density Contrast -0.8'}, xlabel='$log_{10}(R)$R in Mpc', ylabel='$\\frac{1}{V} \\frac{dN_v}{dlnR_v}$'>

../_images/notebooks_application_tutorial_25_1.png

Filter Voids Based on properties

Void finders like ZOBOV need a pre cleaning based on properties. Here we show how to clean this voids using properties.

You can find the properties of the voids in zobov_voids.extra_.properties , this will return a pandas DataFrame with its properties.
You can filter the properties easily with this DataFrame and get the index of the objects you are intrested in.
Use zobov_voids.filter_by_index(index) to retrieve a new filtered Void object.

[11]:

# Dataframe with the properties:
zobov = zobov_voids.e_.properties
zobov.head()

[11]:

	Void#	FileVoid#	CoreParticle	CoreDens	ZoneVol	Zone#Part	Void#Zones	VoidVol	Void#Part	VoidDensContrast	VoidProb	Tracers_in_void
0	1	29182	5853504	0.024460	2899.596	2071	25	11172.46	10015	8.211967	1.510000e-104	[5720094, 5526898, 5530280, 5535112, 5535116, ...
1	2	27800	5504005	0.027375	1827.330	1160	10595	3881550.00	3416629	7.305856	5.020000e-75	[78656, 10479, 10480, 10481, 10484, 14586, 145...
2	3	11721	2419068	0.031486	3843.677	2839	310	116495.50	99375	5.896360	2.680000e-41	[521204, 464275, 477191, 478624, 493395, 50674...
3	4	16154	3280825	0.030940	1422.366	712	1334	542419.70	451554	5.842445	2.850000e-40	[107890, 17446, 17447, 17449, 25171, 25172, 25...
4	5	32370	6546154	0.032252	1176.252	673	190	75163.24	63200	5.701917	1.110000e-37	[1804782, 1775636, 1775637, 1800244, 1775635, ...

[15]:

# Original zobov
zobov_voids

[15]:

<Voids 'ZobovVF' 34228V, 6784817T>

After the cleaning process we achieve less but more meaningful objects:

[14]:

# Filter Properties:
zobov_filtered = zobov[(zobov.VoidProb < 0.1) & (zobov.VoidDensContrast < 1.5)]

zobov_voids_filtered = zobov_voids.filter_by_index(index=zobov_filtered.index)
zobov_voids_filtered

[14]:

<Voids 'ZobovVF' 1362V, 6784817T>