.. _re_correlation:

Modelling Correlation in Random Effects
##########################################
.. include:: re_covariance/intro.txt


Uncorrelated Effects 
======================
.. note:: 
    See the :ref:`sum_link_gen_indep_fit_indep_tut` and :ref:`sum_link_gen_diag_fit_diag_tut` for the :ref:`tut_script` used to generate results in this section. 

We can synthesize observations where parameters are uncorrelated simply by creating, for each parameter, an independent random effect with its own variance:

.. literalinclude:: /_autogen/pop_examples/re_covariance/gen_indep_fit_indep/gen_sections/EFFECTS.pyml
    :language: pyml

Mathematically, this is equivalent to modelling the two parameters jointly via a :term:`covariance matrix` with the individual variances along the diagonal and zeros everywhere else. In |popy| this structure is enforced using the :pyml:`~diag_matrix()` notation:

.. literalinclude:: /_autogen/pop_examples/re_covariance/gen_diag_fit_diag/gen_sections/EFFECTS.pyml
    :language: pyml

for the covariance matrix and using a multivariate normal distribution, :pyml:`~mnorm()`, to model the two random effects as a vector (in this case with zero mean). Throughout this chapter we will continue to use the covariance matrix form rather than independent variables.

With independent inter-subject variability in :pyml:`m[CL]` and :pyml:`m[V]` the generated curves (:numref:`fig_uncorr_effects`) vary widely from individuals with high CL and low V (|ie| a high KE) to individuals with a low CL and high V (|ie| a low KE).

.. figure:: /_autogen/pop_examples/re_covariance/gen_diag_fit_diag/images/gen_sim_grph_outputs/allOBS_vs_TIME/comb_spag.*
    :name: fig_uncorr_effects
    :align: center
    :width: 50%
    
    Simulated prediction curves for a population with independent inter-subject variability on parameters CL and V

In all cases here, we ensure that the generated observations are the same by specifying a fixed :pyml:`rand_seed` option in the |method_options| section of the tutorial script:

.. literalinclude:: /_autogen/pop_examples/re_covariance/gen_diag_fit_diag/gen_sections/METHOD_OPTIONS.pyml
    :language: pyml
      
    
.. uncorr_data_uncorr_fit:

Uncorrelated Fit to Uncorrelated Effects
---------------------------------------------
.. note:: See the :ref:`sum_link_gen_diag_fit_diag_tut` for the :ref:`tut_script` used to generate results in this section. 
.. include:: re_covariance/diag_diag_example.txt


.. uncorr_data_corr_fit:

Correlated Fit to Uncorrelated Effects
-------------------------------------------
.. note:: See the :ref:`sum_link_gen_diag_fit_full_tut` for the :ref:`tut_script` used to generate results in this section. 
.. include:: re_covariance/diag_full_example.txt


Correlated Effects 
===================
.. note:: See the :ref:`sum_link_gen_full_fit_diag_tut` for the :ref:`tut_script` used to generate results in this section. 

We now run the same experiments, only using new observations that have been synthesized from a model where the variability in :pyml:`m[CL]` and :pyml:`m[V]` *is* correlated. This is a more realistic example, since both clearance and volume of distribution can increase with weight.

..  comment 
    What is rotated by 45 degrees??
    
    Specifically, the original generating covariance matrix is "rotated" through 45 degrees:

.. literalinclude:: /_autogen/pop_examples/re_covariance/gen_full_fit_diag/gen_sections/EFFECTS.pyml
    :language: pyml

.. figure:: /_autogen/pop_examples/re_covariance/gen_full_fit_diag/images/gen_sim_grph_outputs/allOBS_vs_TIME/comb_spag.*
    :name: fig_correlated_effects
    :align: center
    :width: 50%   

    Simulated prediction curves for a population with correlated inter-subject variability on parameters CL and V.

The generated predictions (:numref:`fig_correlated_effects`) are qualitatively different from those generated without correlation because the resulting *KE* values have a smaller spread. (Correlated changes in *CL* and *V* cancel out to a degree.)
   

.. corr_data_uncorr_fit:

Uncorrelated Fit to Correlated Effects
-----------------------------------------
.. note:: See the :ref:`sum_link_gen_full_fit_diag_tut` for the :ref:`tut_script` used to generate results in this section. 
.. include:: re_covariance/full_diag_example.txt


.. corr_data_corr_fit:

Correlated Fit to Correlated Effects
-----------------------------------------
.. note:: See the :ref:`sum_link_gen_full_fit_full_tut` for the :ref:`tut_script` used to generate results in this section. 
.. include:: re_covariance/full_full_example.txt
                                                        

These examples illustrate the importance of including the flexibility to capture correlations in random effects where the model parameters vary together (for example, due to a common underlying property). 

Where the observations do not come from a correlated source, having the flexibility to capture correlations has little real benefit to the model fit (a constrained covariance matrix would do just as well) but may require more data to estimate the greater number of parameters; the model should be as complex as it needs to be, but no more.

In practice, it is rare for PK parameters to be completely independent with a correlation of zero. Accounting for correlations tends to lead to better VPCs and more realistic simulations of future patients.
  
.. comment

    Shrinkage
    =============