Tutorial: Binning process with sklearn Pipeline

This example shows how to use a binning process as a transformation within a Scikit-learn Pipeline. A pipeline generally comprises the application of one or more transforms and a final estimator.

[1]:

import numpy as np
import pandas as pd

from optbinning import BinningProcess

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline

To get us started, let’s load a well-known dataset from the UCI repository

[2]:

class Data:
    def __init__(self, data, target, feature_names):
        self.data = data
        self.target = target
        self.feature_names = feature_names


def load_boston():
    data_url = "../../../tests/data/boston_housing.csv"
    raw_df = pd.read_csv(data_url, sep=r"\s+", skiprows=22, header=None)
    raw_data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
    target = raw_df.values[1::2, 2]
    feature_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS',
                     'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT']

    return Data(raw_data, target, feature_names)

data = load_boston()

variable_names = data.feature_names
X = data.data
y = data.target

[3]:

categorical_variables = ['CHAS']

Instantiate a BinningProcess object class with variable names and the list of numerical variables to be considered categorical. Create pipeline object by providing two steps: a binning process transformer and a linear regression estimator.

[4]:

binning_process = BinningProcess(variable_names,
                                 categorical_variables=categorical_variables)

[5]:

lr = Pipeline(steps=[('binning_process', binning_process),
                     ('regressor', LinearRegression())])

Split dataset into train and test Fit pipeline with training data.

[6]:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

[7]:

lr.fit(X_train, y_train)

[7]:

Pipeline(steps=[('binning_process',
                 BinningProcess(categorical_variables=['CHAS'],
                                variable_names=['CRIM', 'ZN', 'INDUS', 'CHAS',
                                                'NOX', 'RM', 'AGE', 'DIS',
                                                'RAD', 'TAX', 'PTRATIO', 'B',
                                                'LSTAT'])),
                ('regressor', LinearRegression())])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Pipeline

?Documentation for PipelineiFitted

Parameters

	steps steps: list of tuples List of (name of step, estimator) tuples that are to be chained in sequential order. To be compatible with the scikit-learn API, all steps must define `fit`. All non-last steps must also define `transform`. See :ref:`Combining Estimators <combining_estimators>` for more details.	[('binning_process', ...), ('regressor', ...)]
	transform_input transform_input: list of str, default=None The names of the :term:`metadata` parameters that should be transformed by the pipeline before passing it to the step consuming it. This enables transforming some input arguments to ``fit`` (other than ``X``) to be transformed by the steps of the pipeline up to the step which requires them. Requirement is defined via :ref:`metadata routing <metadata_routing>`. For instance, this can be used to pass a validation set through the pipeline. You can only set this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.6	None
	memory memory: str or object with the joblib.Memory interface, default=None Used to cache the fitted transformers of the pipeline. The last step will never be cached, even if it is a transformer. By default, no caching is performed. If a string is given, it is the path to the caching directory. Enabling caching triggers a clone of the transformers before fitting. Therefore, the transformer instance given to the pipeline cannot be inspected directly. Use the attribute ``named_steps`` or ``steps`` to inspect estimators within the pipeline. Caching the transformers is advantageous when fitting is time consuming. See :ref:`sphx_glr_auto_examples_neighbors_plot_caching_nearest_neighbors.py` for an example on how to enable caching.	None
	verbose verbose: bool, default=False If True, the time elapsed while fitting each step will be printed as it is completed.	False

BinningProcess

Parameters

	variable_names	['CRIM', 'ZN', ...]
	categorical_variables	['CHAS']
	max_n_prebins	20
	min_prebin_size	0.05
	min_n_bins	None
	max_n_bins	None
	min_bin_size	None
	max_bin_size	None
	max_pvalue	None
	max_pvalue_policy	'consecutive'
	selection_criteria	None
	fixed_variables	None
	special_codes	None
	split_digits	None
	binning_fit_params	None
	binning_transform_params	None
	n_jobs	None
	verbose	False

LinearRegression

?Documentation for LinearRegression

Parameters

	fit_intercept fit_intercept: bool, default=True Whether to calculate the intercept for this model. If set to False, no intercept will be used in calculations (i.e. data is expected to be centered).	True
	copy_X copy_X: bool, default=True If True, X will be copied; else, it may be overwritten.	True
	tol tol: float, default=1e-6 The precision of the solution (`coef_`) is determined by `tol` which specifies the convergence criterion of the underlying solver. `tol` is set as `atol` and `btol` of :func:`scipy.sparse.linalg.lsqr` when fitting on sparse training data. `tol` is set as `cond` of :func:`scipy.linalg.lstsq` when fitting on dense training data. .. versionadded:: 1.7 .. versionchanged:: 1.9 Now supported on dense data, interpreted as the `cond` parameter.	1e-06
	n_jobs n_jobs: int, default=None The number of jobs to use for the computation. This will only provide speedup in case of sufficiently large problems, that is if firstly `n_targets > 1` and secondly `X` is sparse or if `positive` is set to `True`. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary <n_jobs>` for more details.	None
	positive positive: bool, default=False When set to ``True``, forces the coefficients to be positive. This option is only supported for dense arrays. For a comparison between a linear regression model with positive constraints on the regression coefficients and a linear regression without such constraints, see :ref:`sphx_glr_auto_examples_linear_model_plot_nnls.py`. .. versionadded:: 0.24	False

Fitted attributes

Name	Type	Value
coef_ coef_: array of shape (n_features, ) or (n_targets, n_features) Estimated coefficients for the linear regression problem. If multiple targets are passed during the fit (y 2D), this is a 2D array of shape (n_targets, n_features), while if only one target is passed, this is a 1D array of length n_features.	ndarray[float64](13,)	[ 0.22,-0.12,-0.15,..., 0.22, 0.23, 0.48]
intercept_ intercept_: float or array of shape (n_targets,) Independent term in the linear model. Set to 0.0 if `fit_intercept = False`.	float64	-6.95
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	13
rank_ rank_: int Rank of matrix `X`. Only available when `X` is dense.	int64	np.int64(13)
singular_ singular_: array of shape (min(X, y),) Singular values of `X`. Only available when `X` is dense.	ndarray[float64](13,)	[262.97,140.34, 89.66,..., 36.44, 33.77, 23.61]

[8]:

y_test_predict = lr.predict(X_test)

print("MSE:      {:.3f}".format(mean_squared_error(y_test, y_test_predict)))
print("MAE:      {:.3f}".format(mean_absolute_error(y_test, y_test_predict)))
print("R2 score: {:.3f}".format(r2_score(y_test, y_test_predict)))

MSE:      13.602
MAE:      2.490
R2 score: 0.815

In this case, the performance metrics show that the binning process transformation is effective in improving predictions.

[9]:

lr2 = LinearRegression()
lr2.fit(X_train, y_train)

y_test_predict = lr2.predict(X_test)

print("MSE:      {:.3f}".format(mean_squared_error(y_test, y_test_predict)))
print("MAE:      {:.3f}".format(mean_absolute_error(y_test, y_test_predict)))
print("R2 score: {:.3f}".format(r2_score(y_test, y_test_predict)))

MSE:      24.291
MAE:      3.189
R2 score: 0.669

Binning process statistics

The binning process of the pipeline can be retrieved to show information about the problem and timing statistics.

[10]:

binning_process.information(print_level=1)

optbinning (Version 1.0.0)
Copyright (c) 2019-2026 Guillermo Navas-Palencia, Apache License 2.0

  Statistics
    Number of records                    404
    Number of variables                   13
    Target type                   continuous

    Number of numerical                   12
    Number of categorical                  1
    Number of selected                    13

  Time                                1.6123 sec

The summary method returns basic statistics for each binned variable.

[11]:

binning_process.summary()

[11]:

	name	dtype	status	selected	n_bins	woe	quality_score
0	CRIM	numerical	OPTIMAL	True	10	93.949585	0.001846
1	ZN	numerical	OPTIMAL	True	3	62.306865	0.380976
2	INDUS	numerical	OPTIMAL	True	7	75.118213	0.048957
3	CHAS	categorical	OPTIMAL	True	2	52.476876	0.149723
4	NOX	numerical	OPTIMAL	True	7	86.777245	0.076548
5	RM	numerical	OPTIMAL	True	9	104.021845	0.030523
6	AGE	numerical	OPTIMAL	True	9	75.403568	0.002715
7	DIS	numerical	OPTIMAL	True	8	73.097554	0.010709
8	RAD	numerical	OPTIMAL	True	4	62.798802	0.414959
9	TAX	numerical	OPTIMAL	True	6	74.957266	0.198384
10	PTRATIO	numerical	OPTIMAL	True	7	73.905115	0.020726
11	B	numerical	OPTIMAL	True	9	83.013298	0.060997
12	LSTAT	numerical	OPTIMAL	True	12	124.279471	0.001821