{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Advanced constraints in CRFE on superconductor data\n\nRecursive feature elimination with :class:`~tagarnina.crfe.CRFE` for a continuous\ntarget: the critical temperature of 21,263 superconductors. The constraints\ncombine group bounds with implications between features. The verbose output\nshows the progress of the elimination.\n\nThe dataset is public and is available on OpenML (ID 44964), under the CC BY 4.0\nlicence. Please cite it as: Hamidieh, K., A data-driven statistical model for\npredicting the critical temperature of a superconductor, Computational Materials\nScience, 154, 346-354 (2018).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Load the data\nThe target is the critical temperature in kelvin. Each feature is a statistic\n(for example ``mean_``, ``range_`` or ``wtd_entropy_``) of an elemental\nproperty, named by the statistic followed by the property, such as\n``range_Density``. The weighted statistics (``wtd_`` prefix) weight each\nelement by its fraction in the compound.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.datasets import fetch_openml\n\nX, y = fetch_openml(data_id=44964, as_frame=True, return_X_y=True, parser=\"auto\")\n\nprint(X.shape)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Constraints\nFeatures are grouped by their statistic, which is the prefix of the name. A\n:class:`~tagarnina.optimization.constraints.AtMostKConstraint` limits the range\nstatistics, an :class:`~tagarnina.optimization.constraints.AtLeastKConstraint`\nrequires some mean statistics, and a\n:class:`~tagarnina.optimization.constraints.GroupConstraint` sets both bounds on\nthe weighted entropy statistics.\n\nAn :class:`~tagarnina.optimization.constraints.OnlyEnforceIfConstraint` links\neach weighted mean to the unweighted mean of the same property: the weighted\nmean can be selected only if the unweighted mean is too. This is a modelling\nchoice that keeps the two versions of a property together.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from tagarnina.optimization.constraints import (\n    AtLeastKConstraint,\n    AtMostKConstraint,\n    Constraints,\n    GroupConstraint,\n    OnlyEnforceIfConstraint,\n)\n\n\ndef features_with_prefix(prefix):\n    return [c for c in X.columns if c.startswith(prefix)]\n\n\nrange_features = features_with_prefix(\"range_\")\nmean_features = features_with_prefix(\"mean_\")\nwtd_entropy_features = features_with_prefix(\"wtd_entropy_\")\nproperties = [c.removeprefix(\"mean_\") for c in mean_features]\n\nconstraints = Constraints(\n    constraints=[\n        AtMostKConstraint(features=range_features, max_features=3),\n        AtLeastKConstraint(features=mean_features, min_features=2),\n        GroupConstraint(\n            features=wtd_entropy_features,\n            min_features=1,\n            max_features=4,\n        ),\n        *[\n            OnlyEnforceIfConstraint(feature1=f\"wtd_mean_{p}\", feature2=f\"mean_{p}\")\n            for p in properties\n        ],\n    ],\n)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Feature elimination\n:class:`~tagarnina.crfe.CRFE` keeps 15 features. ``verbose=1`` prints the\nsettings, one line per fitted support, and a summary. ``verbose=2`` also prints\nthe details of the solver and the model.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn.ensemble import RandomForestRegressor\n\nfrom tagarnina.crfe import CRFE\n\nestimator = RandomForestRegressor(\n    n_estimators=100,\n    min_samples_leaf=5,\n    n_jobs=-1,\n    random_state=0,\n)\n\ncrfe = CRFE(\n    estimator=estimator,\n    constraints=constraints,\n    step=0.1,\n    n_features_to_select=15,\n    scoring=\"r2\",\n    verbose=1,\n)\ncrfe.fit(X, y)\n\nselected_features = crfe.get_feature_names_out()\nprint(selected_features)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Check the constraints on the selected features. The counts below are\ncompared with the bounds set above.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "selected = set(selected_features)\n\nn_range = sum(c in selected for c in range_features)\nn_mean = sum(c in selected for c in mean_features)\nn_wtd_entropy = sum(c in selected for c in wtd_entropy_features)\nprint(f\"range statistics: {n_range} selected (at most 3)\")\nprint(f\"mean statistics: {n_mean} selected (at least 2)\")\nprint(f\"weighted entropy statistics: {n_wtd_entropy} selected (between 1 and 4)\")\n\nviolations = [\n    p for p in properties if f\"wtd_mean_{p}\" in selected and f\"mean_{p}\" not in selected\n]\nprint(f\"Weighted means without their unweighted mean: {violations}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The run summary records how the elimination ended and how many probes were\ninfeasible.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(crfe.summary_[\"termination\"])\nprint(f\"Features that could not be removed: {crfe.summary_['n_blocked']}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The plots show the score and the importance gap at each support size. The score\nis computed on the training data, so it is optimistic and tends to increase with\nthe number of features. The importance gap is the importance lost because of the\nconstraints, compared with standard RFE with the same number of features. See\nthe basic example for the full history.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n\nn_features = crfe.history_[\"n_features\"]\n\nfig, (ax_score, ax_gap) = plt.subplots(1, 2, figsize=(10, 4))\nax_score.plot(n_features, crfe.history_[\"score\"], marker=\"o\")\nax_score.set_xlabel(\"Number of features\")\nax_score.set_ylabel(\"R2 (training data)\")\nax_score.set_title(\"Score\")\n\nax_gap.plot(n_features, crfe.history_[\"importance_gap\"], marker=\"o\")\nax_gap.set_xlabel(\"Number of features\")\nax_gap.set_ylabel(\"Importance gap\")\nax_gap.set_title(\"Importance lost to constraints\")\nfig.tight_layout()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.13.17"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}