.. _simple_tut_example:


Generate data and Fit using Simple |poppk| Model
#################################################

|popy| provides a method to simulate, analyse and compare results in a single script, which is ideal for generating tutorials or illustrative examples. Here we will demonstrate a :ref:`tut_script` using the same compartment model as used in :ref:`simple_fit_example`, see Figure :numref:`fig_simple_popkp_diagram_tut`:-

.. _fig_simple_popkp_diagram_tut:

.. figure:: /_autogen/quick_start/tut_example1/compartment_diagram.*
    :width: 50%
    :align: center
    
    One compartment model with bolus dosing for :ref:`tut_script`. Here |camt| is the amount of the bolus dose. |mke| is the elimination rate and |scentral| is the current amount in the central compartment.
     
.. note::
    
   See the :ref:`sum_link_tut_example1_tut` obtained by the |popy| developers for this example, including input script and output data file.
     
A :ref:`tut_script` can be used as a theoretical tool to investigate identifiability of |pkpd| models, because the true |fx| parameters and underlying structure of the data are known. Unfortunately this is never the case in a real life analysis.
 
This documentation makes extensive use of :ref:`tut_scripts<tut_script>` to create examples to illustrate different :ref:`indiv_pkpd_index`.
 
 
.. _running_a_simple_tut_script:
 
Running the Tutorial Script
============================

This tutorial example requires a single input file, ``c:\PoPy\examples\tut_example1.pyml``
                            
:ref:`open_a_popy_command_prompt` to set up the |popy| environment in the folder ``c:\PoPy\examples\``

With the |popy| environment enabled, you can call :ref:`popy_run` on the :ref:`tut_script` from the command line:-

.. code-block:: console

    popy run tut_example1.pyml

When the tut script has completed, you can view the output of the fit using :ref:`popy_view`, by typing the following command:-

.. code-block:: console

    popy view tut_example1.pyml.html

Note the extra '.html' extension in the above command. This command opens a local .html file in your web browser to summarise the result of the generating process.

You can compare your local html output with the pre-computed documentation output, see :ref:`sum_link_tut_example1_tut`. You should expect some minor numerical differences when comparing results with the documentation.

.. _tut_syntax_simple:

Syntax of Tut Script
========================

The major structural difference between a :ref:`gen_script` or :ref:`fit_script` and a :ref:`tut_script` is that the tut_script has separate |gen_effects| and |fit_effects| sections to describe both the generating and fitting effects. The |gen_effects| section for this tutorial example is as follows:-

.. literalinclude:: 
    /_autogen/quick_start/tut_example1/tut_sections/GEN_EFFECTS.pyml
    :language: pyml      

The |fit_effects| section for this tutorial example is as follows:-

.. literalinclude:: 
    /_autogen/quick_start/tut_example1/tut_sections/FIT_EFFECTS.pyml
    :language: pyml      

The |gen_effects| get copied into the :ref:`gen_script` and renamed |effects|. Similarly the |fit_effects| get copied into the :ref:`fit_script` and also renamed |effects|. From the examples above you can see that the GEN_EFFECTS->POP section has:-

.. code-block:: pyml

    f[X] = true_value
    
Whereas the FIT_EFFECTS->POP section has:-

.. code-block:: pyml

    f[X] ~ unif(0.001, 100) starting_value
    
Reflecting the fact that the |fx| are known constants for a :ref:`gen_script`, but are unknown values to be estimated in a :ref:`fit_script`, with lower and upper limits of [0.001, 100]. Note in |popy| you can also use 'P' meaning positive, as a shortcut for '~unif(0.0,+inf)'.

The GEN_EFFECTS->POP level has this extra line:-

.. code-block:: pyml
    
    c[AMT] = 100.0
    
This sets the dose amount to be 100.0 units for all individuals. 

The GEN_EFFECTS->ID level also contains these extra lines:-

.. code-block:: pyml

    c[ID] = sequential(20)
    t[DOSE] = 1.0
    t[OBS] ~ unif(1.0, 50.0; 5)
               
These lines are passed to the :ref:`gen_script` and generate 20 individuals all with a dose at time 1.0 and 5 observations uniformally sampled in the time interval [1.0, 50.0]. See :ref:`example_two_level_effects_gen` for more information on the :ref:`gen_script` syntax within a |effects| section.

.. _summary_of_simple_tut_results:

Summary of Tut Results
=============================

See :ref:`sum_link_tut_example1_tut` for example |html| outputs generated by the |popy| developers.

On your local machine, the :ref:`tut_script` generates an output folder containing four new scripts:-

.. code-block::

    simple_tut_example.pyml_output/
        simple_tut_example_gen.pyml
        simple_tut_example_fit.pyml
        simple_tut_example_comp.pyml
        simple_tut_example_tutsum.pyml
                                              
See :ref:`files_generated_by_tut_script` for more info. The purpose of each of theses scripts is as follows:-

.. _table_tut_output_scripts_simple:

.. list-table:: Scripts output by a tutorial script
    :header-rows: 1
    
    * - Script
      - Purpose
      - Documentation

    * - \*_gen.pyml
      - Generate synthetic data set from model
      - :ref:`gen_script`

    * - \*_fit.pyml
      - Fit model to synthetic data set
      - :ref:`fit_script`
      
    * - \*_comp.pyml
      - Compare gen model and fit model to synthetic data set
      - :ref:`comp_script`

    * - \*_tutsum.pyml
      - Summary of generating and fitting and comparison results
      - :ref:`tutsum_script`

These four scripts are run in order. 

The :ref:`gen_script` uses a |effects| structure similar to a :ref:`fit_script`, but with some extra commands to generate new data rows, see :ref:`tut_syntax_simple` above. Alternatively see :ref:`builtin_gen_example`, for a longer explanation of how a :ref:`gen_script` works.

The :ref:`fit_script` here is very similar to the |pkpd| model described in :ref:`simple_fit_example`. Therefore here we will focus on the :ref:`comp_script` outputs. To generate the comp output, you need this entry in your :ref:`tut_script` |OUTPUT_SCRIPTS| section:-

.. code-block:: pyml

    OUTPUT_SCRIPTS:
        COMP: {output_mode: run}
    
Otherwise the :ref:`tutsum_script` will have **no** comp output to summarise. The comp outputs are |pk| curves from the fitted and generated |fx| parameters and the associated :term:`objective function` values. The simplest :ref:`comp_script` output is a visual comparison of the true and fitted |fx| |pk| curves and the synthetic generated data, see :numref:`table_true_vs_fit_fx_plots_simple`.

.. _table_true_vs_fit_fx_plots_simple:

.. list-table:: Fitted model |pk| curves vs true model |pk| curves for first three individuals 
    :class: image_table
    
    * - .. thumbnail:: /_autogen/quick_start/tut_example1/images/fit_sim_grph_outputs/indOBS_vs_TIME/000001.*
      - .. thumbnail:: /_autogen/quick_start/tut_example1/images/fit_sim_grph_outputs/indOBS_vs_TIME/000002.*
      - .. thumbnail:: /_autogen/quick_start/tut_example1/images/fit_sim_grph_outputs/indOBS_vs_TIME/000003.* 

The solid blue lines in :numref:`table_true_vs_fit_fx_plots_simple` show the predicted |pk| curves for the fitted model |fx| values. The dotted blue lines show the |pk| curves for the true |fx| values that were used to generated the data set (in the :ref:`gen_script`). The blue dots are the target |cdvcen| values from the data file.

The target |cdvcen| values have measurement noise added, so blue dot data points do |not| lie exactly on the true |fx| curves. The graphs show that the |pk| curves for the fitted |fx| are almost identical to the true |fx| curves, this is to be expected as the model only contains a single model parameter |mke| and we have 5 observations per individual. 

If the :ref:`comp_script` has been run, the :ref:`tutsum_script` outputs convenient tables to compare the initial, fitted and true |fx| values, see :numref:`table_true_vs_fit_fx_main_values_simple`, :numref:`table_true_vs_fit_fx_var_values_simple` and :numref:`table_true_vs_fit_fx_noise_values_simple`.

.. _table_true_vs_fit_fx_main_values_simple:

.. csv-table:: Comparison of initial, fitted and true |fke| values
    :file: ../../../_autogen/quick_start/tut_example1/fx_comp_main.csv
    :header-rows: 1

.. _table_true_vs_fit_fx_var_values_simple:

.. csv-table:: Comparison of initial, fitted and true |fke_isv| values
    :file: ../../../_autogen/quick_start/tut_example1/fx_comp_variance.csv
    :header-rows: 1
    
.. _table_true_vs_fit_fx_noise_values_simple:

.. csv-table:: Comparison of initial, fitted and true |fpnoise| values
    :file: ../../../_autogen/quick_start/tut_example1/fx_comp_noise.csv
    :header-rows: 1

:numref:`table_true_vs_fit_fx_main_values_simple` shows that the |fke| parameter is recovered reasonably well, in the sense that the fitted value 0.106 is close to the true generating value 0.1 starting from an initial value of 0.05. Similarly the fitted |fke_isv| and |fpnoise| parameters are close to the true generating values.
    
The objective function :ref:`comp_script` computes the :term:`objective function` given the synthetic data and the true generating |fx| parameters (the |rx| are re-optimised). In this case the true |fx| |objv| is:-

.. code-block:: pyml

    -44.20

The :ref:`comp_script` also computes he |objv| for the fitted |fx| and optimised |rx|, which is as follows:-
    
.. code-block:: pyml

    -48.43
      
The lower objective value for the fitted |fx| is quite common, because the fitted |fx| can take advantage of noise in the generated synthetic data set. If the size of the synthetic data set is increased, then it is likely that the |fke|, |fke_isv| and |fpnoise| parameters will move closer to the true generating values and the |objvs| will also converge.
      
In this simple example the parameters are very easy to identify. For a more challenging example see :ref:`builtin_tut_example`.
      
.. _simple_mtut_example:
      
Generate multiple data sets and Fit using Simple |poppk| Model
==================================================================

The tutorial example above generates a single data set from user specified true |fx| values. In |popy| it is possible to generalise this approach and sample true |fx| values multiple times to create multiple data sets. Then fit the same model to each data set.

In this section we walk through how to generate multiple data sets using a :ref:`mtut_script`, using the same simple one compartment model as shown in :numref:`fig_simple_popkp_diagram_tut`.

.. _running_a_mtut_script_simple:
 
Running the MTut Script
------------------------

..
    duplication of text w.r.t. docs for builtin_mtut_example

This multi tutorial example makes use of a single script file, ``c:\PoPy\examples\mtut_example1.pyml``
                            
:ref:`open_a_popy_command_prompt` to setup the |popy| environment in the folder ``c:\PoPy\examples\``

With the |popy| environment enabled, you can call :ref:`popy_run` on the :ref:`mtut_script` from the command line:-

.. code-block:: console

    popy run mtut_example1.pyml
    
to run the script.

Running a :ref:`mtut_script` can take a considerable amount of time, as it is equivalent to running a :ref:`tut_script` multiple times. 
However in this toy example we only run the fit/gen cycle 30 times and only a small number of the |fx| parameters are estimated.

.. _syntax_of_mtut_script:

Syntax of MTut Script
------------------------

The :ref:`mtut_script` specifies the number of populations to sample as follows:-

.. code-block:: pyml

    OUTPUT_OPTIONS: {n_pop_samples: 30}

The :ref:`mtut_script` encodes **both** the data generation and fitting in the |gen_effects| and |fit_effects| sections, like a :ref:`tut_script`. The syntax is the same. In this example the |gen_effects| section is as follows:-

.. literalinclude:: simple_mtut_gen_effects.pyml
    :language: pyml      

And the |fit_effects| section is as follows:-

.. literalinclude:: simple_mtut_fit_effects.pyml
    :language: pyml      

Here the generated values of |fke|, |fpnoise| and |fke_isv| are sampled from uniform distributions. The fitting process is initialised with (constant) values in the centre of the uniform distribution, used to sample the generating |fe| values.
    
.. _summary_of_simple_mtut_results:

Summary of MTut Results
-------------------------------

The :ref:`mtut_script` should generate an output folder containing three new scripts:-

.. code-block::

    mtut_example1.pyml_output/
        mtut_example1_mgen.pyml
        mtut_example1_mfit.pyml
        mtut_example1_mcomp.pyml
                                              
The purpose of each of theses scripts is as follows:-

.. _table_mtut_output_scripts_simple:

.. list-table:: Scripts output by a multi tutorial script
    :header-rows: 1
    
    * - Script
      - Purpose
      - Documentation

    * - \*_mgen.pyml
      - Generate multiple synthetic data sets from model
      - :ref:`mgen_script`

    * - \*_mfit.pyml
      - Fit model to multiple synthetic data sets
      - :ref:`mfit_script`

    * - \*_mcomp.pyml
      - Compare gen model and fit model |fx|
      - :ref:`mcomp_script`
 
The :ref:`mgen_script` is very similar to the :ref:`gen_script` described in :ref:`tut_syntax_simple` and the :ref:`mfit_script` is very similar to the :ref:`fit_script` described in :ref:`simple_fit_example`. See :ref:`files_generated_by_mtut_script` for more info. 
   
Here we mainly discuss the :ref:`mcomp_script` outputs, which processes the results of :ref:`mgen_script` and :ref:`mfit_script`. The simplest output is a visual comparison of the true and fitted |fx| values as shown in :numref:`table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple`.

.. _table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple:

.. list-table:: Fitted model vs true scatter plots for |fke|, |fke_isv| and |fpnoise|
    :class: image_table
    
    * - .. thumbnail:: /_autogen/quick_start/mtut_example1/fx_scatter/fitted_vs_true_for_f_ke_.*
      - .. thumbnail:: /_autogen/quick_start/mtut_example1/fx_scatter/fitted_vs_true_for_f_ke_isv_.*
      - .. thumbnail:: /_autogen/quick_start/mtut_example1/fx_scatter/fitted_vs_true_for_f_pnoise_.*
      
In :numref:`table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple` the blue dots are a scatter plot of fitted |fx| vs true |fx|. The green dots are initial |fx| vs true |fx|. For example in the case of |fke| the true values are sampled as follows:-

.. code-block:: pyml

    f[KE] ~ unif(0.05,0.15)
    
|ie| the true values are uniformly sampled in the range [0.05,0.15]. The fitted |fke| parameters are modelled as follows:-
    
.. code-block:: pyml

    f[KE] ~ P0.1
 
The initial values for |fke| are always 0.1, see green dots in a horizontal line on the left graph in :numref:`table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple`. The 'P' specifies that the fitting value of |fke| is restricted to positive numbers. The final fitting values are the blue dots on the left graph in :numref:`table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple`. For |fke| the blue dots are clustered along the black 45 degree line. Hence fitting for |fke| works well, this agrees with the initial findings in :ref:`simple_fit_example`. The blue dots for |fke_isv| and |fpnoise| (centre and right graphs in :numref:`table_true_vs_mfit_fx_plots_builtin_f_ke_f_ke_isv_fpnoise_simple`) are not as tightly clustered around the 45 degree line, indicating these parameters are harder to identify than |fke|. However both |fke_isv| and |fpnoise| show a reasonable correlation between the true and fitted values.
      
Note it might well be possible to carry out a more statistical analysis of correlation between the true and fitted |fx|, for example finding a line of best fit through the scatter plot data. The :ref:`mcomp_script` outputs are all saved to \.csv files, see :ref:`files_generated_by_mcomp_script`, which could easily be loaded into |r| or other statistical packages for further analysis. 

.. only:: browser

    .. _next_steps_simple_tut:

    Next Steps
    ================== 

    You can see more complicated examples of using :ref:`tut_script` and :ref:`mtut_script`:-

    * :ref:`builtin_tut_example`
    * :ref:`builtin_mtut_example` 

    Or see :ref:`typical_workflows` for a general overview of using |popy| for |pkpd| modelling.
