birl.benchmark module¶
General benchmark template for all registration methods. It also serves for evaluating the input registration pairs (while no registration is performed, there is only the initial deformation)
Sample run (usage):
mkdir ./results
python benchmarks/bm_registration.py -t data_images/pairs-imgs-lnds_histol.csv -d ./data_images -o ./results --unique
Copyright (C) 2016-2019 Jiri Borovec <jiri.borovec@fel.cvut.cz>
-
class
birl.benchmark.
ImRegBenchmark
(params)[source]¶ Bases:
birl.utilities.experiments.Experiment
General benchmark class for all registration methods. It also serves for evaluating the input registration pairs.
Parameters: params (dict) – dictionary with experiment configuration, the required options are names in REQUIRED_PARAMS, note that the basic parameters are inherited The benchmark has following steps:
check all necessary paths and required parameters
load cover file and set all paths as absolute
run individual registration experiment in sequence or in parallel (nb_workers > 1); if the particular experiment folder exist (assume completed experiment) and skip it
- create experiment folder and init experiment
- generate execution command
- run the command (an option to lock it in single thread)
- evaluate experiment, set the expected outputs and visualisation
- clean all extra files if any
visualise results abd evaluate registration results
Note
The actual implementation simulates the “IDEAL” registration while it blindly copies the reference landmarks as results of the registration. In contrast to the right registration, it copies the moving images so there is alignment (consistent warping) between resulting landmarks and image.
Examples
>>> # Running in single thread: >>> from birl.utilities.data_io import create_folder, update_path >>> path_out = create_folder('temp_results') >>> path_csv = os.path.join(update_path('data_images'), 'pairs-imgs-lnds_mix.csv') >>> params = {'path_table': path_csv, ... 'path_out': path_out, ... 'nb_workers': 1, ... 'unique': False, ... 'visual': True} >>> benchmark = ImRegBenchmark(params) >>> benchmark.run() True >>> del benchmark >>> shutil.rmtree(path_out, ignore_errors=True)
>>> # Running in multiple parallel threads: >>> from birl.utilities.data_io import create_folder, update_path >>> path_out = create_folder('temp_results') >>> path_csv = os.path.join(update_path('data_images'), 'pairs-imgs-lnds_mix.csv') >>> params = {'path_table': path_csv, ... 'path_out': path_out, ... 'nb_workers': 2, ... 'unique': False, ... 'visual': True} >>> benchmark = ImRegBenchmark(params) >>> benchmark.run() True >>> del benchmark >>> shutil.rmtree(path_out, ignore_errors=True)
initialise benchmark
Parameters: params (dict) – parameters -
_ImRegBenchmark__check_exist_regist
(idx, path_dir_reg)[source]¶ check whether the particular experiment already exists and have results
if the folder with experiment already exist and it is also part of the loaded finished experiments, sometimes the oder may mean failed experiment
Parameters: Return bool:
-
_ImRegBenchmark__execute_method
(method, input_table, path_csv=None, desc='', aggr_experiments=False, nb_workers=None)[source]¶ execute a method in sequence or parallel
Parameters: Returns:
-
_ImRegBenchmark__export_df_experiments
(path_csv=None)[source]¶ export the DataFrame with registration results
Parameters: | None path_csv (str) – path to output CSV file
-
_ImRegBenchmark__images_preprocessing
(item)[source]¶ create some pre-process images, convert to gray scale and histogram matching
Parameters: item (dict) – the input record Return dict: updated item with optionally added pre-process images
-
_ImRegBenchmark__remove_pproc_images
(item)[source]¶ remove preprocess (temporary) image if they are not also final
Parameters: item (dict) – the input record Return dict: updated item with optionally removed temp images
-
_absolute_path
(path, destination='data', base_path=None)[source]¶ update te path to the dataset or output
Parameters: Return str: updated path
-
classmethod
_clear_after_registration
(item)[source]¶ clean unnecessarily files after the registration
Parameters: item (dict) – dictionary with regist. information Return dict: the same or updated regist. info
-
_copy_config_to_expt
(field_path)[source]¶ copy particular configuration to the experiment folder
Parameters: field_path (str) – field from parameters containing a path to file
-
_execute_img_registration
(item)[source]¶ execute the image registration itself
Parameters: item (dict) – record Return dict: record
-
_extract_execution_time
(item)[source]¶ if needed update the execution time
Parameters: item (dict) – dictionary {str: value} with registration params Return float|None: time in minutes
-
classmethod
_extract_warped_image_landmarks
(item)[source]¶ get registration results - warped registered images and landmarks
Parameters: item (dict) – dictionary with registration params Return dict: paths to warped images/landmarks
-
_generate_regist_command
(item)[source]¶ generate the registration command(s)
Parameters: item (dict) – dictionary with registration params Return str|list(str): the execution commands
-
_get_paths
(item, prefer_pproc=True)[source]¶ expand the relative paths to absolute, if TEMP path is used, take it
Parameters: Return tuple(str,str,str,str): path to reference and moving image and reference and moving landmarks
-
classmethod
_image_diag
(item, path_img_ref=None)[source]¶ - get the image diagonal from several sources
- diagonal exists in the table
- image size exist in the table
- reference image exists
Parameters: - item (dict|DF) – one row from the table
- path_img_ref (str) – optional path to the reference image
Return float|None: image diagonal
-
classmethod
_load_warped_image
(item, path_experiment=None)[source]¶ load the wapted image if it exists
Parameters: - item (dict) – row with the experiment
- path_experiment (str|None) – path to the experiment folder
Return ndarray:
-
_parse_regist_results
(item)[source]¶ evaluate rests of the experiment and identity the registered image and landmarks when the process finished
Parameters: item (dict) – dictionary {str: value} with registration params Return dict:
-
_perform_registration
(df_row)[source]¶ run single registration experiment with all sub-stages
Parameters: df_row (tuple(int,dict)) – row from iterated table
-
classmethod
_prepare_img_registration
(item)[source]¶ prepare the experiment folder if it is required, eq. copy some extra files
Parameters: item (dict) – dictionary with regist. params Return dict: the same or updated registration info
-
_relativize_path
(path, destination='path_exp')[source]¶ extract relative path according given parameter
Parameters: Return str: relative or the original path
-
classmethod
_visual_image_move_warp_lnds_move_warp
(item, path_dataset=None, path_experiment=None)[source]¶ visualise the case with warped moving image and landmarks to the reference frame so they are simple to overlap
Parameters: - item (dict) – row with the experiment
- path_dataset (str|None) – path to the dataset folder
- path_experiment (str|None) – path to the experiment folder
Return obj|None:
-
classmethod
_visual_image_move_warp_lnds_ref_warp
(item, path_dataset=None, path_experiment=None)[source]¶ visualise the case with warped reference landmarks to the move frame
Parameters: - item (dict) – row with the experiment
- path_dataset (str|None) – path to the dataset folder
- path_experiment (str|None) – path to the experiment folder
Return obj|None:
-
classmethod
compute_registration_accuracy
(df_experiments, idx, points1, points2, state='', img_diag=None, wo_affine=False)[source]¶ compute statistic on two points sets
IRE - Initial Registration Error TRE - Target Registration Error
Parameters: - df_experiments (DF) – DataFrame with experiments
- idx (int) – index of tha particular record
- points1 (ndarray) – np.array<nb_points, dim>
- points2 (ndarray) – np.array<nb_points, dim>
- state (str) – whether it was before of after registration
- img_diag (float) – target image diagonal
- wo_affine (bool) – without affine transform, assume only local/elastic deformation
-
classmethod
compute_registration_statistic
(idx_row, df_experiments, path_dataset=None, path_experiment=None, path_reference=None)[source]¶ after successful registration load initial nad estimated landmarks afterwords compute various statistic for init, and final alignment
Parameters:
-
classmethod
main
(params=None)[source]¶ run the Main of selected experiment
Parameters: - cls – class of selected benchmark
- params (dict) – set of input parameters
-
classmethod
visualise_registration
(idx_row, path_dataset=None, path_experiment=None)[source]¶ visualise the registration results according what landmarks were estimated - in registration or moving frame
Parameters:
-
COL_IMAGE_EXT_TEMP
= ' TEMP'[source]¶ extension to the image column name for temporary pre-process image
-
COL_NB_LANDMARKS_INPUT
= 'nb. dataset landmarks'[source]¶ number of landmarks in dataset (min of moving and reference)
-
COL_POINTS_MOVE_WARP
= 'Warped source landmarks'[source]¶ moving landmarks warped to the reference frame
-
COL_POINTS_REF_WARP
= 'Warped target landmarks'[source]¶ reference landmarks warped to the moving frame
-
COL_ROBUSTNESS
= 'Robustness'[source]¶ define robustness as improved image alignment from initial state
-
COL_TIME_PREPROC
= 'Pre-processing time [minutes]'[source]¶ measured time of image pre-processing in minutes
-
COVER_COLUMNS_EXT
= ('Target image', 'Source image', 'Target landmarks', 'Source landmarks', 'Image size [pixels]', 'Image diagonal [pixels]')[source]¶
-
COVER_COLUMNS_WRAP
= ('Target image', 'Source image', 'Target landmarks', 'Source landmarks', 'Warped target image', 'Warped source image', 'Warped target landmarks', 'Warped source landmarks')[source]¶
-
EXECUTE_TIMEOUT
= 3600[source]¶ does not work for Py2
Type: timeout for executing single image registration, NOTE
-
NAME_IMAGE_MOVE_WARP_POINTS
= 'image_warped_landmarks_warped.jpg'[source]¶ output image name in experiment folder for reg. results - image and landmarks are warped
-
NAME_IMAGE_REF_POINTS_WARP
= 'image_ref_landmarks_warped.jpg'[source]¶ output image name in experiment folder for reg. results - warped landmarks in reference image
-
NAME_IMAGE_REF_WARP
= 'image_refence-warped.jpg'[source]¶ output image name in experiment folder for reg. results - overlap of reference and warped image
-
NAME_IMAGE_WARPED_VISUAL
= 'registration_visual_landmarks.jpg'[source]¶ output image name in experiment folder for showing improved alignment by used registration
-
NAME_RESULTS_CSV
= 'results-summary.csv'[source]¶ default file for exporting results in table format
-
birl.benchmark.
_df_drop_unnamed
(df)[source]¶ Drop columns was index without name and was loaded as Unnamed: 0.
-
birl.benchmark.
export_summary_results
(df_experiments, path_out, params=None, name_txt='results-summary.txt', name_csv='results-summary.csv')[source]¶ export the summary as CSV and TXT
Parameters: >>> export_summary_results(pd.DataFrame(), '')
-
birl.benchmark.
filter_paired_landmarks
(item, path_dataset, path_reference, col_source, col_target)[source]¶ filter all relevant landmarks which were used and copy them to experiment
- The case is that in certain challenge stage users had provided just a subset
- of all image landmarks which could be laos shuffled. The idea is to filter identify all user used (provided in dataset) landmarks and filter them from temporary reference dataset.
Parameters: Return tuple(float,ndarray,ndarray): match ratio, filtered ref and move landmarks
>>> p_data = update_path('data_images') >>> p_csv = os.path.join(p_data, 'pairs-imgs-lnds_histol.csv') >>> df = pd.read_csv(p_csv) >>> ratio, lnds_ref, lnds_move = filter_paired_landmarks(dict(df.iloc[0]), p_data, p_data, ... ImRegBenchmark.COL_POINTS_MOVE, ImRegBenchmark.COL_POINTS_REF) >>> ratio 1.0 >>> lnds_ref.shape == lnds_move.shape True