{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Parametric Analysis with `MorphingIterator` (v4 Update)\n", "\n", "This notebook demonstrates the most powerful feature of the `pyfwg` library: the `MorphingIterator` class. \n", "\n", "This class is designed for running **parametric studies**, where you need to execute the morphing process multiple times with different parameters (e.g., different GCMs, interpolation methods, or even different EPW files). It uses a Pandas DataFrame to define all the runs in a structured and easy-to-manage way.\n", "\n", "**Note:** `pyfwg` now fully supports **FutureWeatherGenerator v4.0.x** (Global) and **v2.0.x** (Europe), which include a new keyword-based CLI, support for string-based parameter options, and additional output formats (like `'MET'` or `'CSV'`)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Iterator Workflow\n", "\n", "The iterator is designed to be used in a clear, multi-step process that provides control and visibility at each stage:\n", "\n", "1. **Instantiate the Iterator**: Choose which tool you want to work with (`MorphingWorkflowGlobal` or `MorphingWorkflowEurope`).\n", "2. **Set Default Values**: Define parameters that are common to all runs in your batch.\n", "3. **Define the Runs DataFrame**: Specify the parameters that will change for each run, using either an Excel file or a Pandas DataFrame directly.\n", "4. **Generate the Plan & Prepare Workflows**: A single command that applies all defaults, validates the plan, and prepares all the workflow instances for execution.\n", "5. **Execute the Workflows**: A final command to run the entire batch of simulations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 1: Instantiate the Iterator\n", "\n", "First, we import the necessary classes and create an instance of the `MorphingIterator`. We must tell it which workflow class it will be managing. In this example, we'll use `MorphingWorkflowGlobal`." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-31T07:49:54.409973Z", "start_time": "2025-12-31T07:49:53.771645Z" } }, "source": [ "import pandas as pd\n", "import os\n", "from pyfwg import MorphingIterator, MorphingWorkflowGlobal, export_template_to_excel, load_runs_from_excel, get_available_lczs, DEFAULT_GLOBAL_GCMS\n", "\n", "# We specify that we want to use the Global tool for all runs.\n", "iterator = MorphingIterator(workflow_class=MorphingWorkflowGlobal)" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-31 08:49:54 - INFO - MorphingIterator initialized for MorphingWorkflowGlobal.\u001B[0m\n" ] } ], "execution_count": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 2: Pre-flight Checks & Common Parameters\n", "\n", "Before defining a large batch of runs, it's good practice to check which parameters are valid. Here, we'll get the available LCZs for our weather files. Then, we'll use `set_default_values` to define parameters that will be the same for all runs, such as the JAR path and the LCZs we just validated." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-31T07:51:16.573898Z", "start_time": "2025-12-31T07:50:48.180831Z" } }, "source": [ "# !!! IMPORTANT: Update this path to your V4 JAR !!!\n", "jar_path = r\"D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\"\n", "epw_files_dir = 'epws/wo_pattern'\n", "epw_files = [os.path.join(epw_files_dir, f) for f in os.listdir(epw_files_dir) if f.endswith('.epw')]\n", "\n", "# Get available LCZs for each EPW file to ensure our choices are valid.\n", "available_lczs = get_available_lczs(\n", " epw_paths=epw_files,\n", " fwg_jar_path=jar_path,\n", ")" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-31 08:50:48 - INFO - --- Fetching available LCZs for 2 EPW file(s) ---\u001B[0m\n", "\u001B[32m2025-12-31 08:50:48 - INFO - Checking LCZ pair (Original: 0, Target: 0) availability for GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw...\u001B[0m\n", "\u001B[32m2025-12-31 08:50:48 - INFO - --- Applying UHI effect to GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw ---\u001B[0m\n", "\u001B[32m2025-12-31 08:50:48 - INFO - Executing command: java -jar \"D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\" -u -epw=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpi4zkf7rl\\GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw -output_folder=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpi4zkf7rl\\ -uhi=true:0:0 -output_type=EPW\u001B[0m\n", "\u001B[32m2025-12-31 08:51:02 - INFO - Available LCZs for 'GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw': [1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17]\u001B[0m\n", "\u001B[32m2025-12-31 08:51:02 - INFO - Checking LCZ pair (Original: 0, Target: 0) availability for sevilla_in_this_one_the_uhi_is_type-1.epw...\u001B[0m\n", "\u001B[32m2025-12-31 08:51:02 - INFO - --- Applying UHI effect to sevilla_in_this_one_the_uhi_is_type-1.epw ---\u001B[0m\n", "\u001B[32m2025-12-31 08:51:02 - INFO - Executing command: java -jar \"D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\" -u -epw=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpilqkeeuq\\sevilla_in_this_one_the_uhi_is_type-1.epw -output_folder=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpilqkeeuq\\ -uhi=true:0:0 -output_type=EPW\u001B[0m\n", "\u001B[32m2025-12-31 08:51:16 - INFO - Available LCZs for 'sevilla_in_this_one_the_uhi_is_type-1.epw': [2, 3, 6, 8, 9, 11, 12, 13, 14, 16]\u001B[0m\n" ] } ], "execution_count": 2 }, { "metadata": {}, "cell_type": "markdown", "source": "The available LCZs for the epw files are:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-12-31T07:52:07.698523Z", "start_time": "2025-12-31T07:52:07.682253Z" } }, "cell_type": "code", "source": "print(available_lczs)", "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw': [1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17], 'sevilla_in_this_one_the_uhi_is_type-1.epw': [2, 3, 6, 8, 9, 11, 12, 13, 14, 16]}\n" ] } ], "execution_count": 3 }, { "metadata": {}, "cell_type": "code", "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-30 10:30:11 - INFO - --- Fetching available LCZs for 2 EPW file(s) ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:11 - INFO - Checking LCZ pair (Original: 0, Target: 0) availability for GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:11 - INFO - --- Applying UHI effect to GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:11 - INFO - Executing command: java -jar \"D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\" -u -epw=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmps4z54yza\\GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw -output_folder=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmps4z54yza\\ -uhi=true:0:0 -output_type=EPW\u001B[0m\n", "\u001B[32m2025-12-30 10:30:26 - INFO - Available LCZs for 'GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw': [2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17]\u001B[0m\n", "\u001B[32m2025-12-30 10:30:26 - INFO - Checking LCZ pair (Original: 0, Target: 0) availability for sevilla_in_this_one_the_uhi_is_type-1.epw...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:26 - INFO - --- Applying UHI effect to sevilla_in_this_one_the_uhi_is_type-1.epw ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:26 - INFO - Executing command: java -jar \"D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\" -u -epw=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpyvd_1evq\\sevilla_in_this_one_the_uhi_is_type-1.epw -output_folder=C:\\Users\\sanga\\AppData\\Local\\Temp\\tmpyvd_1evq\\ -uhi=true:0:0 -output_type=EPW\u001B[0m\n", "\u001B[32m2025-12-30 10:30:40 - INFO - Available LCZs for 'sevilla_in_this_one_the_uhi_is_type-1.epw': [3, 6, 8, 9, 11, 12, 13, 14, 16]\u001B[0m\n" ] } ], "execution_count": 2, "source": [ "\n", "# Define the keyword mapping rules that will be used for all runs in this batch.\n", "mapping_rules = {\n", " 'city': {\n", " 'seville': ['sevilla', 'SVQ'],\n", " 'london': ['london', 'gatwick']\n", " },\n", " 'uhi': {\n", " 'type-1': 'type-1',\n", " 'type-2': 'type-2'\n", " }\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### `set_default_values` Parameters\n", "\n", "This method allows you to define parameters that are common to all runs in your batch, so you don't have to repeat them in every row of your DataFrame. All arguments are optional.\n", "\n", "##### Workflow Control Parameters\n", "* `final_output_dir` (`str`): A default output directory for runs that don't specify one.\n", "* `output_filename_pattern` (`str`): A default renaming pattern. **Must contain** `{ssp}`/`{rcp}` and `{year}`.\n", "* `scenario_mapping` (`Dict`): A default dictionary for mapping scenario names.\n", "* `fwg_jar_path` (`str`): The path to the Future Weather Generator `.jar` file.\n", "* `run_incomplete_files` (`bool`, default: `False`): If `True`, also processes partially categorized files.\n", "* `delete_temp_files` (`bool`, default: `True`): If `True`, deletes temporary folders after processing.\n", "* `temp_base_dir` (`str`): A default base directory for temporary files.\n", "* `fwg_show_tool_output` (`bool`, default: `False`): If `True`, prints the FWG tool's console output in real-time.\n", "* `fwg_params` (`Dict`, default: `None`): A dictionary for base FWG parameters.\n", "\n", "##### Future Weather Generator Tool Parameters\n", "* `fwg_gcms` (`List[str]`, default: `None`): **For Global tool only.** A default list of GCMs to use.\n", "* `fwg_rcm_pairs` (`List[str]`, default: `None`): **For Europe tool only.** A default list of GCM-RCM pairs to use.\n", "* `fwg_create_ensemble` (`bool`, default: `True`): If `True`, creates an ensemble from the selected models.\n", "* `fwg_winter_sd_shift` (`float`, default: `0.0`): Winter standard deviation shift.\n", "* `fwg_summer_sd_shift` (`float`, default: `0.0`): Summer standard deviation shift.\n", "* `fwg_month_transition_hours` (`int`, default: `72`): Hours for month transition.\n", "* `fwg_use_multithreading` (`bool`, default: `True`): Use multithreading.\n", "* `fwg_interpolation_method_id` (`Union[int, str]`, default: `0`): Interpolation method ID or string (e.g., `'AVG4P'` for V4).\n", "* `fwg_limit_variables` (`bool`, default: `True`): Limit variables to physical bounds.\n", "* `fwg_solar_hour_adjustment` (`Union[int, str]`, default: `1`): Solar hour adjustment option (e.g., `'By_Month'` for V4).\n", "* `fwg_diffuse_irradiation_model` (`Union[int, str]`, default: `1`): Diffuse irradiation model option (e.g., `'Engerer_2015'` for V4).\n", "* `fwg_add_uhi` (`bool`, default: `True`): Add UHI effect.\n", "* `fwg_epw_original_lcz` (`int`, default: `14`): Original EPW LCZ.\n", "* `fwg_target_uhi_lcz` (`int`, default: `1`): Target UHI LCZ.\n", "* `fwg_output_type` (`str`, default: `'EPW'`): **V4 feature.** Format of output files: `'EPW'`, `'MET'`, or `'CSV'`.\n", "* `fwg_version` (`Optional[str]`, default: `None`): Force a specific version CLI (e.g., `'4'`)." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:41.095462Z", "start_time": "2025-12-30T09:30:41.088965Z" } }, "source": [ "# Set the default values that will be common to all runs.\n", "iterator.set_default_values(\n", " fwg_jar_path=jar_path,\n", " output_filename_pattern='{city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year}',\n", " # We just checked that LCZs 2 and 3 are available in all epws, so we can use them as defaults.\n", " fwg_epw_original_lcz=2,\n", " fwg_target_uhi_lcz=3,\n", " fwg_version='4' # We are using v4\n", ")" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-30 10:30:41 - INFO - Custom default values have been set for the iterator: {'output_filename_pattern': '{city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year}', 'fwg_jar_path': 'D:\\\\OneDrive - Universidad de Cádiz (uca.es)\\\\Programas\\\\FutureWeatherGenerator_v4.0.2.jar', 'fwg_epw_original_lcz': 2, 'fwg_target_uhi_lcz': 3, 'fwg_version': '4'}\u001B[0m\n" ] } ], "execution_count": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 3: Define the Runs DataFrame\n", "\n", "This is where you specify what will change between each run. You have three flexible options for this." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 3.1: Option A - Using an Excel Template\n", "\n", "This is the recommended approach for non-programmers or for managing a large number of runs. You can export a blank template, fill it in with your scenarios, and load it back into `pyfwg`." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:42.187780Z", "start_time": "2025-12-30T09:30:41.165267Z" } }, "source": [ "# 1. Export the blank template to an Excel file.\n", "template_path = 'my_parametric_study.xlsx'\n", "export_template_to_excel(iterator, file_path=template_path)\n", "\n", "print(f\"--- Original Template Structure (saved to '{template_path}') ---\")\n", "# Read and display the empty template to show the available columns.\n", "pd.read_excel(template_path)" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-30 10:30:41 - INFO - Generating Excel template for MorphingWorkflowGlobal...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:41 - INFO - Template successfully exported to 'D:\\Python\\pyfwg\\docs\\source\\tutorials\\my_parametric_study.xlsx'\u001B[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "--- Original Template Structure (saved to 'my_parametric_study.xlsx') ---\n" ] }, { "data": { "text/plain": [ "Empty DataFrame\n", "Columns: [epw_paths, input_filename_pattern, keyword_mapping, final_output_dir, output_filename_pattern, scenario_mapping, fwg_jar_path, run_incomplete_files, delete_temp_files, temp_base_dir, fwg_show_tool_output, fwg_params, fwg_gcms, fwg_create_ensemble, fwg_winter_sd_shift, fwg_summer_sd_shift, fwg_month_transition_hours, fwg_use_multithreading, fwg_interpolation_method_id, fwg_limit_variables, fwg_solar_hour_adjustment, fwg_diffuse_irradiation_model, fwg_add_uhi, fwg_epw_original_lcz, fwg_target_uhi_lcz, fwg_output_type, fwg_version]\n", "Index: []\n", "\n", "[0 rows x 27 columns]" ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsinput_filename_patternkeyword_mappingfinal_output_diroutput_filename_patternscenario_mappingfwg_jar_pathrun_incomplete_filesdelete_temp_filestemp_base_dir...fwg_use_multithreadingfwg_interpolation_method_idfwg_limit_variablesfwg_solar_hour_adjustmentfwg_diffuse_irradiation_modelfwg_add_uhifwg_epw_original_lczfwg_target_uhi_lczfwg_output_typefwg_version
\n", "

0 rows × 27 columns

\n", "
" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 4 }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:42.419240Z", "start_time": "2025-12-30T09:30:42.402531Z" } }, "source": [ "# --- At this point, you would open 'my_parametric_study.xlsx', ---\n", "# --- fill in the rows with your scenarios, and save it. ---\n", "# --- For this example, we assume you saved it as 'my_parametric_study_modified.xlsx' ---" ], "outputs": [], "execution_count": 5 }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:43.041665Z", "start_time": "2025-12-30T09:30:42.916641Z" } }, "source": [ "# 2. Load the completed scenarios from your edited Excel file.\n", "template_path_mod = 'my_parametric_study_modified.xlsx'\n", "runs_from_excel = load_runs_from_excel(template_path_mod)\n", "\n", "print(\"\\n--- Runs Loaded from Excel (Data types are now correct) ---\")\n", "runs_from_excel" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-30 10:30:42 - INFO - Loading runs from 'my_parametric_study_modified.xlsx'...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:42 - INFO - Runs loaded and data types converted successfully.\u001B[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "--- Runs Loaded from Excel (Data types are now correct) ---\n" ] }, { "data": { "text/plain": [ " epw_paths input_filename_pattern \\\n", "0 epws/wo_pattern/GBR_London.Gatwick.037760_IWEC... NaN \n", "1 epws/wo_pattern/sevilla_in_this_one_the_uhi_is... NaN \n", "\n", " keyword_mapping final_output_dir output_filename_pattern \\\n", "0 NaN results_using_excel/london NaN \n", "1 NaN results_using_excel/seville NaN \n", "\n", " scenario_mapping fwg_jar_path run_incomplete_files delete_temp_files \\\n", "0 NaN NaN NaN NaN \n", "1 NaN NaN NaN NaN \n", "\n", " temp_base_dir ... fwg_summer_sd_shift fwg_month_transition_hours \\\n", "0 NaN ... NaN NaN \n", "1 NaN ... NaN NaN \n", "\n", " fwg_use_multithreading fwg_interpolation_method_id fwg_limit_variables \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "\n", " fwg_solar_hour_adjustment fwg_diffuse_irradiation_model fwg_add_uhi \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "\n", " fwg_epw_original_lcz fwg_target_uhi_lcz \n", "0 NaN NaN \n", "1 NaN NaN \n", "\n", "[2 rows x 25 columns]" ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsinput_filename_patternkeyword_mappingfinal_output_diroutput_filename_patternscenario_mappingfwg_jar_pathrun_incomplete_filesdelete_temp_filestemp_base_dir...fwg_summer_sd_shiftfwg_month_transition_hoursfwg_use_multithreadingfwg_interpolation_method_idfwg_limit_variablesfwg_solar_hour_adjustmentfwg_diffuse_irradiation_modelfwg_add_uhifwg_epw_original_lczfwg_target_uhi_lcz
0epws/wo_pattern/GBR_London.Gatwick.037760_IWEC...NaNNaNresults_using_excel/londonNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
1epws/wo_pattern/sevilla_in_this_one_the_uhi_is...NaNNaNresults_using_excel/sevilleNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
\n", "

2 rows × 25 columns

\n", "
" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 6 }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 3.2: Option B - Using a Pandas DataFrame Directly\n", "\n", "For quick tests or when working within a script, you can define your runs programmatically." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:43.337237Z", "start_time": "2025-12-30T09:30:43.299556Z" } }, "source": [ "# Get the blank template DataFrame.\n", "runs_df_direct = iterator.get_template_dataframe()\n", "\n", "# --- Run 1: Run the first EPW file with one set of GCMs ---\n", "runs_df_direct.loc[0] = {\n", " 'epw_paths': epw_files[0],\n", " 'final_output_dir': './results/0',\n", " 'fwg_gcms': ['CanESM5']\n", "}\n", "\n", "# --- Run 2: Run the second EPW file with a different GCM ---\n", "runs_df_direct.loc[1] = {\n", " 'epw_paths': epw_files[1],\n", " 'final_output_dir': './results/1',\n", " 'fwg_gcms': ['MIROC6']\n", "}\n", "\n", "print(\"--- Runs Defined Directly in Pandas ---\")\n", "runs_df_direct" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Runs Defined Directly in Pandas ---\n" ] }, { "data": { "text/plain": [ " epw_paths input_filename_pattern \\\n", "0 epws/wo_pattern\\GBR_London.Gatwick.037760_IWEC... NaN \n", "1 epws/wo_pattern\\sevilla_in_this_one_the_uhi_is... NaN \n", "\n", " keyword_mapping final_output_dir output_filename_pattern \\\n", "0 NaN ./results/0 NaN \n", "1 NaN ./results/1 NaN \n", "\n", " scenario_mapping fwg_jar_path run_incomplete_files delete_temp_files \\\n", "0 NaN NaN NaN NaN \n", "1 NaN NaN NaN NaN \n", "\n", " temp_base_dir ... fwg_use_multithreading fwg_interpolation_method_id \\\n", "0 NaN ... NaN NaN \n", "1 NaN ... NaN NaN \n", "\n", " fwg_limit_variables fwg_solar_hour_adjustment \\\n", "0 NaN NaN \n", "1 NaN NaN \n", "\n", " fwg_diffuse_irradiation_model fwg_add_uhi fwg_epw_original_lcz \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "\n", " fwg_target_uhi_lcz fwg_output_type fwg_version \n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "\n", "[2 rows x 27 columns]" ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsinput_filename_patternkeyword_mappingfinal_output_diroutput_filename_patternscenario_mappingfwg_jar_pathrun_incomplete_filesdelete_temp_filestemp_base_dir...fwg_use_multithreadingfwg_interpolation_method_idfwg_limit_variablesfwg_solar_hour_adjustmentfwg_diffuse_irradiation_modelfwg_add_uhifwg_epw_original_lczfwg_target_uhi_lczfwg_output_typefwg_version
0epws/wo_pattern\\GBR_London.Gatwick.037760_IWEC...NaNNaN./results/0NaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
1epws/wo_pattern\\sevilla_in_this_one_the_uhi_is...NaNNaN./results/1NaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
\n", "

2 rows × 27 columns

\n", "
" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 7 }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Step 3.3: Option C - Hybrid Approach (Load from Excel, then Modify)\n", "\n", "You can also combine the two methods. Here, we load the runs from our Excel file and then programmatically add a new run to the DataFrame." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:43.976785Z", "start_time": "2025-12-30T09:30:43.927144Z" } }, "source": [ "first_gcm = list(DEFAULT_GLOBAL_GCMS)[0]\n", "runs_df_hybrid = runs_from_excel.copy() if 'runs_from_excel' in locals() else runs_df_direct.copy()\n", "\n", "# Add a new row (run) to the DataFrame.\n", "runs_df_hybrid.loc[len(runs_df_hybrid)] = {\n", " 'epw_paths': epw_files[1],\n", " 'final_output_dir': 'results_hybrid/seville',\n", " 'fwg_gcms': [first_gcm]\n", "}\n", "\n", "print(\"--- Final DataFrame for Execution (Hybrid) ---\")\n", "runs_df_hybrid" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Final DataFrame for Execution (Hybrid) ---\n" ] }, { "data": { "text/plain": [ " epw_paths input_filename_pattern \\\n", "0 epws/wo_pattern/GBR_London.Gatwick.037760_IWEC... NaN \n", "1 epws/wo_pattern/sevilla_in_this_one_the_uhi_is... NaN \n", "2 epws/wo_pattern\\sevilla_in_this_one_the_uhi_is... NaN \n", "\n", " keyword_mapping final_output_dir output_filename_pattern \\\n", "0 NaN results_using_excel/london NaN \n", "1 NaN results_using_excel/seville NaN \n", "2 NaN results_hybrid/seville NaN \n", "\n", " scenario_mapping fwg_jar_path run_incomplete_files delete_temp_files \\\n", "0 NaN NaN NaN NaN \n", "1 NaN NaN NaN NaN \n", "2 NaN NaN NaN NaN \n", "\n", " temp_base_dir ... fwg_summer_sd_shift fwg_month_transition_hours \\\n", "0 NaN ... NaN NaN \n", "1 NaN ... NaN NaN \n", "2 NaN ... NaN NaN \n", "\n", " fwg_use_multithreading fwg_interpolation_method_id fwg_limit_variables \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "2 NaN NaN NaN \n", "\n", " fwg_solar_hour_adjustment fwg_diffuse_irradiation_model fwg_add_uhi \\\n", "0 NaN NaN NaN \n", "1 NaN NaN NaN \n", "2 NaN NaN NaN \n", "\n", " fwg_epw_original_lcz fwg_target_uhi_lcz \n", "0 NaN NaN \n", "1 NaN NaN \n", "2 NaN NaN \n", "\n", "[3 rows x 25 columns]" ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsinput_filename_patternkeyword_mappingfinal_output_diroutput_filename_patternscenario_mappingfwg_jar_pathrun_incomplete_filesdelete_temp_filestemp_base_dir...fwg_summer_sd_shiftfwg_month_transition_hoursfwg_use_multithreadingfwg_interpolation_method_idfwg_limit_variablesfwg_solar_hour_adjustmentfwg_diffuse_irradiation_modelfwg_add_uhifwg_epw_original_lczfwg_target_uhi_lcz
0epws/wo_pattern/GBR_London.Gatwick.037760_IWEC...NaNNaNresults_using_excel/londonNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
1epws/wo_pattern/sevilla_in_this_one_the_uhi_is...NaNNaNresults_using_excel/sevilleNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
2epws/wo_pattern\\sevilla_in_this_one_the_uhi_is...NaNNaNresults_hybrid/sevilleNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
\n", "

3 rows × 25 columns

\n", "
" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 8 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 4: Generate the Morphing Workflows and Execution Plan\n", "\n", "This single method is the core of the planning phase. It takes your DataFrame of runs, applies all defaults, maps the file categories, and prepares all the underlying workflow instances.\n", "\n", "Crucially, it also performs a **robust validation check** to prevent accidental file overwrites. It simulates all final output filenames and, if any collisions are found, it provides a comprehensive report of all conflicting files, making it much easier to debug your `output_filename_pattern`.\n", "\n", "The final, detailed execution plan is stored in the `iterator.morphing_workflows_plan_df` attribute for you to review.\n", "\n", "#### `generate_morphing_workflows` Parameters\n", "\n", "This is the core planning method. It takes your DataFrame of runs and prepares everything for execution.\n", "\n", "* `runs_df` (`pd.DataFrame`): **Required.** The user's DataFrame of runs, where each row represents a unique configuration.\n", "* `input_filename_pattern` (`Optional[str]`, default: `None`): A regex pattern for filename mapping, applied as a default to *every* run unless overridden in the DataFrame.\n", "* `keyword_mapping` (`Optional[Dict]`, default: `None`): A dictionary of keyword rules for filename mapping, applied as a default to *every* run unless overridden in the DataFrame.\n", "* `raise_on_overwrite` (`bool`, default: `True`): Controls the behavior when a filename collision is detected. If `True` (the default and recommended setting), the method will raise a `ValueError` to stop execution and prevent data loss. If `False`, it will only print a warning and allow the process to continue." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:44.398403Z", "start_time": "2025-12-30T09:30:44.324352Z" } }, "source": [ "# The mapping strategy is a static argument for the whole batch.\n", "iterator.generate_morphing_workflows(\n", " runs_df=runs_df_hybrid,\n", " keyword_mapping=mapping_rules\n", ")" ], "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001B[32m2025-12-30 10:30:44 - INFO - Generating detailed execution plan and preparing workflows...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern/GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw': {'city': 'london', 'uhi': 'type-2'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern/sevilla_in_this_one_the_uhi_is_type-1.epw': {'city': 'seville', 'uhi': 'type-1'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern\\sevilla_in_this_one_the_uhi_is_type-1.epw': {'city': 'seville', 'uhi': 'type-1'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Validating for potential filename overwrites...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Filename validation passed. No overwrites detected.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Preparing 3 workflow instances...\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern/GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw': {'city': 'london', 'uhi': 'type-2'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 2: Configuring and Previewing Morphing Plan ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern/sevilla_in_this_one_the_uhi_is_type-1.epw': {'city': 'seville', 'uhi': 'type-1'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 2: Configuring and Previewing Morphing Plan ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 1: Mapping categories from filenames ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Mapped 'epws/wo_pattern\\sevilla_in_this_one_the_uhi_is_type-1.epw': {'city': 'seville', 'uhi': 'type-1'}\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Category mapping complete.\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - --- Step 2: Configuring and Previewing Morphing Plan ---\u001B[0m\n", "\u001B[32m2025-12-30 10:30:44 - INFO - Execution plan generated and 3 workflows prepared.\u001B[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "============================================================\n", " MORPHING CONFIGURATION & PREVIEW\n", "============================================================\n", " - FWG JAR Path: D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\n", " - Final Output Directory: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\n", " - EPWs to be Morphed (1 files):\n", " - GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw\n", "\n", " For input file: GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw\n", " -> Generated 'ssp126_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp126_2050.epw\n", " -> Generated 'ssp245_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp245_2050.epw\n", " -> Generated 'ssp370_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp370_2050.epw\n", " -> Generated 'ssp585_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp585_2050.epw\n", " -> Generated 'ssp126_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp126_2080.epw\n", " -> Generated 'ssp245_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp245_2080.epw\n", " -> Generated 'ssp370_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp370_2080.epw\n", " -> Generated 'ssp585_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\london\\london_type-2_gcm-['CanESM5']_ssp585_2080.epw\n", "============================================================\n", "Configuration set. Call execute_morphing() to start the process.\n", "\n", "============================================================\n", " MORPHING CONFIGURATION & PREVIEW\n", "============================================================\n", " - FWG JAR Path: D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\n", " - Final Output Directory: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\n", " - EPWs to be Morphed (1 files):\n", " - sevilla_in_this_one_the_uhi_is_type-1.epw\n", "\n", " For input file: sevilla_in_this_one_the_uhi_is_type-1.epw\n", " -> Generated 'ssp126_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp126_2050.epw\n", " -> Generated 'ssp245_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp245_2050.epw\n", " -> Generated 'ssp370_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp370_2050.epw\n", " -> Generated 'ssp585_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp585_2050.epw\n", " -> Generated 'ssp126_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp126_2080.epw\n", " -> Generated 'ssp245_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp245_2080.epw\n", " -> Generated 'ssp370_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp370_2080.epw\n", " -> Generated 'ssp585_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_using_excel\\seville\\seville_type-1_gcm-['MIROC6']_ssp585_2080.epw\n", "============================================================\n", "Configuration set. Call execute_morphing() to start the process.\n", "\n", "============================================================\n", " MORPHING CONFIGURATION & PREVIEW\n", "============================================================\n", " - FWG JAR Path: D:\\OneDrive - Universidad de Cádiz (uca.es)\\Programas\\FutureWeatherGenerator_v4.0.2.jar\n", " - Final Output Directory: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\n", " - EPWs to be Morphed (1 files):\n", " - sevilla_in_this_one_the_uhi_is_type-1.epw\n", "\n", " For input file: sevilla_in_this_one_the_uhi_is_type-1.epw\n", " -> Generated 'ssp126_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp126_2050.epw\n", " -> Generated 'ssp245_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp245_2050.epw\n", " -> Generated 'ssp370_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp370_2050.epw\n", " -> Generated 'ssp585_2050.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp585_2050.epw\n", " -> Generated 'ssp126_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp126_2080.epw\n", " -> Generated 'ssp245_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp245_2080.epw\n", " -> Generated 'ssp370_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp370_2080.epw\n", " -> Generated 'ssp585_2080.epw' will be moved to: D:\\Python\\pyfwg\\docs\\source\\tutorials\\results_hybrid\\seville\\seville_type-1_gcm-['EC_Earth3']_ssp585_2080.epw\n", "============================================================\n", "Configuration set. Call execute_morphing() to start the process.\n" ] } ], "execution_count": 9 }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:45.423112Z", "start_time": "2025-12-30T09:30:45.397944Z" } }, "source": [ "print(\"--- Detailed Execution Plan ---\")\n", "# Display key columns to verify the plan, including the new category columns.\n", "display_cols = [\n", " 'epw_paths',\n", " 'final_output_dir',\n", " 'fwg_gcms',\n", " 'cat_city',\n", " 'cat_uhi'\n", "]\n", "iterator.morphing_workflows_plan_df[display_cols]" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--- Detailed Execution Plan ---\n" ] }, { "data": { "text/plain": [ " epw_paths \\\n", "0 epws/wo_pattern/GBR_London.Gatwick.037760_IWEC... \n", "1 epws/wo_pattern/sevilla_in_this_one_the_uhi_is... \n", "2 epws/wo_pattern\\sevilla_in_this_one_the_uhi_is... \n", "\n", " final_output_dir fwg_gcms cat_city cat_uhi \n", "0 results_using_excel/london [CanESM5] [london] [type-2] \n", "1 results_using_excel/seville [MIROC6] [seville] [type-1] \n", "2 results_hybrid/seville [EC_Earth3] [seville] [type-1] " ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsfinal_output_dirfwg_gcmscat_citycat_uhi
0epws/wo_pattern/GBR_London.Gatwick.037760_IWEC...results_using_excel/london[CanESM5][london][type-2]
1epws/wo_pattern/sevilla_in_this_one_the_uhi_is...results_using_excel/seville[MIROC6][seville][type-1]
2epws/wo_pattern\\sevilla_in_this_one_the_uhi_is...results_hybrid/seville[EC_Earth3][seville][type-1]
\n", "
" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 10 }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:45.672168Z", "start_time": "2025-12-30T09:30:45.631966Z" } }, "source": [ "# At this point the default values have been applied, and therefore the plan can be fully checked before running the morphing\n", "iterator.morphing_workflows_plan_df" ], "outputs": [ { "data": { "text/plain": [ " epw_paths input_filename_pattern \\\n", "0 epws/wo_pattern/GBR_London.Gatwick.037760_IWEC... None \n", "1 epws/wo_pattern/sevilla_in_this_one_the_uhi_is... None \n", "2 epws/wo_pattern\\sevilla_in_this_one_the_uhi_is... None \n", "\n", " keyword_mapping cat_city cat_uhi \\\n", "0 {'city': {'seville': ['sevilla', 'SVQ'], 'lond... [london] [type-2] \n", "1 {'city': {'seville': ['sevilla', 'SVQ'], 'lond... [seville] [type-1] \n", "2 {'city': {'seville': ['sevilla', 'SVQ'], 'lond... [seville] [type-1] \n", "\n", " final_output_dir output_filename_pattern \\\n", "0 results_using_excel/london {city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year} \n", "1 results_using_excel/seville {city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year} \n", "2 results_hybrid/seville {city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year} \n", "\n", " scenario_mapping fwg_jar_path \\\n", "0 NaN D:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr... \n", "1 NaN D:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr... \n", "2 NaN D:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr... \n", "\n", " run_incomplete_files ... fwg_use_multithreading \\\n", "0 False ... True \n", "1 False ... True \n", "2 False ... True \n", "\n", " fwg_interpolation_method_id fwg_limit_variables fwg_solar_hour_adjustment \\\n", "0 0 True 1 \n", "1 0 True 1 \n", "2 0 True 1 \n", "\n", " fwg_diffuse_irradiation_model fwg_add_uhi fwg_epw_original_lcz \\\n", "0 1 True 2 \n", "1 1 True 2 \n", "2 1 True 2 \n", "\n", " fwg_target_uhi_lcz fwg_output_type fwg_version \n", "0 3 EPW 4 \n", "1 3 EPW 4 \n", "2 3 EPW 4 \n", "\n", "[3 rows x 29 columns]" ], "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epw_pathsinput_filename_patternkeyword_mappingcat_citycat_uhifinal_output_diroutput_filename_patternscenario_mappingfwg_jar_pathrun_incomplete_files...fwg_use_multithreadingfwg_interpolation_method_idfwg_limit_variablesfwg_solar_hour_adjustmentfwg_diffuse_irradiation_modelfwg_add_uhifwg_epw_original_lczfwg_target_uhi_lczfwg_output_typefwg_version
0epws/wo_pattern/GBR_London.Gatwick.037760_IWEC...None{'city': {'seville': ['sevilla', 'SVQ'], 'lond...[london][type-2]results_using_excel/london{city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year}NaND:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr...False...True0True11True23EPW4
1epws/wo_pattern/sevilla_in_this_one_the_uhi_is...None{'city': {'seville': ['sevilla', 'SVQ'], 'lond...[seville][type-1]results_using_excel/seville{city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year}NaND:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr...False...True0True11True23EPW4
2epws/wo_pattern\\sevilla_in_this_one_the_uhi_is...None{'city': {'seville': ['sevilla', 'SVQ'], 'lond...[seville][type-1]results_hybrid/seville{city}_{uhi}_gcm-{fwg_gcms}_{ssp}_{year}NaND:\\OneDrive - Universidad de Cádiz (uca.es)\\Pr...False...True0True11True23EPW4
\n", "

3 rows × 29 columns

\n", "
" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 11 }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also inspect the prepared workflow instances themselves." ] }, { "cell_type": "code", "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:30:46.077339Z", "start_time": "2025-12-30T09:30:46.065556Z" } }, "source": [ "print(\"\\n--- Inspecting first prepared workflow ---\")\n", "if iterator.prepared_workflows:\n", " first_workflow = iterator.prepared_workflows[0]\n", " print(f\"Is config valid? {first_workflow.is_config_valid}\")\n", " print(f\"Files to be morphed: {[os.path.basename(p) for p in first_workflow.epws_to_be_morphed]}\")\n", "else:\n", " print(\"No workflows were prepared, likely due to errors in the plan.\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "--- Inspecting first prepared workflow ---\n", "Is config valid? True\n", "Files to be morphed: ['GBR_London.Gatwick.037760_IWEC_uhi_type-2.epw']\n" ] } ], "execution_count": 12 }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 5: Execute the Morphing Workflows\n", "\n", "This is the final step. The `run_morphing_workflows` method takes no arguments to define the runs, as it simply executes the workflows that were prepared in the previous step." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### `run_morphing_workflows` Parameters\n", "\n", "* `show_tool_output` (`Optional[bool]`, default: `None`): A flag to globally override the console output setting for all workflows in this specific batch execution.\n", " * If set to `True` or `False`, it will **force** this behavior for all runs, ignoring the `fwg_show_tool_output` value in the plan.\n", " * If left as `None` (the default), each run will use the `fwg_show_tool_output` value that was defined for it in the execution plan." ] }, { "metadata": {}, "cell_type": "code", "outputs": [], "execution_count": null, "source": "iterator.run_morphing_workflows(show_tool_output=False)" }, { "metadata": {}, "cell_type": "markdown", "source": "Let's have a look at the files we just generated:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:42:12.295141Z", "start_time": "2025-12-30T09:42:12.284500Z" } }, "cell_type": "code", "source": [ "output_paths = [i for i in iterator.morphing_workflows_plan_df['final_output_dir']]\n", "\n", "for output_path in output_paths:\n", " print(f\"\\n--- Files in '{output_path}' ---\")\n", " for file in os.listdir(output_path):\n", " print(file)" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['results_using_excel/london', 'results_using_excel/seville', 'results_hybrid/seville']\n", "\n", "--- Files in 'results_using_excel/london' ---\n", "london_type-2_gcm-['CanESM5']_ssp126_2050.epw\n", "london_type-2_gcm-['CanESM5']_ssp126_2050.stat\n", "london_type-2_gcm-['CanESM5']_ssp126_2080.epw\n", "london_type-2_gcm-['CanESM5']_ssp126_2080.stat\n", "london_type-2_gcm-['CanESM5']_ssp245_2050.epw\n", "london_type-2_gcm-['CanESM5']_ssp245_2050.stat\n", "london_type-2_gcm-['CanESM5']_ssp245_2080.epw\n", "london_type-2_gcm-['CanESM5']_ssp245_2080.stat\n", "london_type-2_gcm-['CanESM5']_ssp370_2050.epw\n", "london_type-2_gcm-['CanESM5']_ssp370_2050.stat\n", "london_type-2_gcm-['CanESM5']_ssp370_2080.epw\n", "london_type-2_gcm-['CanESM5']_ssp370_2080.stat\n", "london_type-2_gcm-['CanESM5']_ssp585_2050.epw\n", "london_type-2_gcm-['CanESM5']_ssp585_2050.stat\n", "london_type-2_gcm-['CanESM5']_ssp585_2080.epw\n", "london_type-2_gcm-['CanESM5']_ssp585_2080.stat\n", "\n", "--- Files in 'results_using_excel/seville' ---\n", "seville_type-1_gcm-['MIROC6']_ssp126_2050.epw\n", "seville_type-1_gcm-['MIROC6']_ssp126_2050.stat\n", "seville_type-1_gcm-['MIROC6']_ssp126_2080.epw\n", "seville_type-1_gcm-['MIROC6']_ssp126_2080.stat\n", "seville_type-1_gcm-['MIROC6']_ssp245_2050.epw\n", "seville_type-1_gcm-['MIROC6']_ssp245_2050.stat\n", "seville_type-1_gcm-['MIROC6']_ssp245_2080.epw\n", "seville_type-1_gcm-['MIROC6']_ssp245_2080.stat\n", "seville_type-1_gcm-['MIROC6']_ssp370_2050.epw\n", "seville_type-1_gcm-['MIROC6']_ssp370_2050.stat\n", "seville_type-1_gcm-['MIROC6']_ssp370_2080.epw\n", "seville_type-1_gcm-['MIROC6']_ssp370_2080.stat\n", "seville_type-1_gcm-['MIROC6']_ssp585_2050.epw\n", "seville_type-1_gcm-['MIROC6']_ssp585_2050.stat\n", "seville_type-1_gcm-['MIROC6']_ssp585_2080.epw\n", "seville_type-1_gcm-['MIROC6']_ssp585_2080.stat\n", "\n", "--- Files in 'results_hybrid/seville' ---\n", "seville_type-1_gcm-['EC_Earth3']_ssp126_2050.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp126_2050.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp126_2080.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp126_2080.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp245_2050.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp245_2050.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp245_2080.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp245_2080.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp370_2050.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp370_2050.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp370_2080.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp370_2080.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp585_2050.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp585_2050.stat\n", "seville_type-1_gcm-['EC_Earth3']_ssp585_2080.epw\n", "seville_type-1_gcm-['EC_Earth3']_ssp585_2080.stat\n" ] } ], "execution_count": 18 }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cleaning up\n", "\n", "Once the workflow is finished, you can delete the temporary folders created during the process if you don't need them anymore. If you did not enable `delete_temp_files=True` in `configure_and_preview`, you can do it manually.\n", "\n", "**Note on Analysis Files:** In FutureWeatherGenerator v4 (Global) and v2 (Europe), the tool generates several analysis folders (e.g., `00_logs`, `01_processing_transcripts`, etc.) when enabled. These folders are **not** moved to the final output directory; they remain in the temporary directory for inspection. If you delete the temporary directory, these files will be lost." ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-12-30T09:42:55.849452Z", "start_time": "2025-12-30T09:42:55.822217Z" } }, "cell_type": "code", "source": [ "import shutil\n", "import os\n", "\n", "# Clean up the results folders\n", "for folder in ['./morphing_temp_results', './morphing_temp_results_europe', './results_hybrid', './results_using_excel', './00_logs']:\n", " if os.path.exists(folder):\n", " shutil.rmtree(folder)\n", " print(f\"Deleted: {folder}\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted: ./morphing_temp_results\n", "Deleted: ./results_hybrid\n", "Deleted: ./results_using_excel\n", "Deleted: ./00_logs\n" ] } ], "execution_count": 19 }, { "metadata": {}, "cell_type": "code", "outputs": [], "execution_count": null, "source": "" } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 4 }