a OfD@stddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZddlmZddlZdd lmZmZdd lmZdd lmZmZdd lmZmZmZmZdd lm Z m!Z!m"Z"erddlm#Z#ej$ej%ej&ej'ej(ej)ej)dZ*ej&ej+dfej)ej,e fej$ej-dfej%ej-dfej'ej-dfej.ej,dfej(ej/dfiZ0ej-dej+dej,diZ1GdddeZ2dS)) annotations) TYPE_CHECKINGAnyN) infer_dtype)iNaT)NoBufferPresent)cache_readonly)BaseMaskedDtype) ArrowDtypeDatetimeTZDtype)is_string_dtype) PandasBufferPandasBufferPyarrow)Column ColumnBuffersColumnNullType DtypeKind) ArrowCTypes Endiannessdtype_to_arrow_c_fmt)Buffer)iufbUMmzThis column is non-nullablezThis column uses NaN as nullz!This column uses a sentinel valuec@seZdZdZd0ddddddZd d d d Zed d d dZedd ddZ dd ddZ eddZ eddZ ed d ddZ edd ddZd d ddZd1d d!d"d#Zd$d d%d&Zd'd d(d)Zd*d d+d,Zd-d d.d/ZdS)2 PandasColumna A column object, with only the methods and properties required by the interchange protocol defined. A column can contain one or more chunks. Each chunk can contain up to three buffers - a data buffer, a mask buffer (depending on null representation), and an offsets buffer (if variable-size binary; e.g., variable-length strings). Note: this Column object can only be produced by ``__dataframe__``, so doesn't need its own version or ``__column__`` protocol. Tz pd.SeriesboolNone)column allow_copyreturncCsNt|tjrtd|jdt|tjs>tdt|d||_||_ dS)zu Note: doesn't deal with extension arrays yet, just assume a regular Series/ndarray for now. zExpected a Series, got a DataFrame. This likely happened because you called __dataframe__ on a DataFrame which, after converting column names to string, resulted in duplicated names: zD. Please rename these columns before using the interchange protocol.zColumns of type  not handled yetN) isinstancepdZ DataFrame TypeErrorcolumnsSeriesNotImplementedErrortype_col _allow_copy)selfr"r#r0W/var/www/ai-form-bot/venv/lib/python3.9/site-packages/pandas/core/interchange/column.py__init__Ts  zPandasColumn.__init__int)r$cCs|jjS)z2 Size of the column, in elements. )r-sizer/r0r0r1r4hszPandasColumn.sizecCsdS)z7 Offset of first element. Always zero. rr0r5r0r0r1offsetnszPandasColumn.offsetztuple[DtypeKind, int, str, str]cCs|jj}t|tjrB|jjj}||j\}}}}tj ||t j fSt |rvt |jdvrltjdt|t j fStdn ||SdS)N)stringemptyz.Non-string object dtypes are not supported yet)r-dtyper&r'ZCategoricalDtypevaluescodes_dtype_from_pandasdtyper CATEGORICALrNATIVEr rSTRINGrr+)r/r:r<_ZbitwidthZc_arrow_dtype_f_strr0r0r1r:vs.    zPandasColumn.dtypecCst|jd}|dur&td|dt|tr:|jj}n.t|trN|j j}nt|t rb|jj}n|j}|dkr||j t j |fS||j dt||fS)z/ See `self.dtype` for details. N Data type z& not supported by interchange protocolz bool[pyarrow]r9) _NP_KINDSgetkind ValueErrorr&r Z numpy_dtype byteorderr baser itemsizerBOOLr)r/r:rErGr0r0r1r=s"      z$PandasColumn._dtype_from_pandasdtypecCs:|jdtjkstd|jjjdtt |jjj dS)a: If the dtype is categorical, there are two options: - There are only values in the data buffer. - There is a separate non-categorical Column encoding for categorical values. Raises TypeError if the dtype is not categorical Content of returned dict: - "is_ordered" : bool, whether the ordering of dictionary indices is semantically meaningful. - "is_dictionary" : bool, whether a dictionary-style mapping of categorical values to other objects exists - "categories" : Column representing the (implicit) mapping of indices to category values (e.g. an array of cat1, cat2, ...). None if not a dictionary-style categorical. rzCdescribe_categorical only works on a column with categorical dtype!T)Z is_orderedZ is_dictionary categories) r:rr>r(r-catorderedrr'r*rKr5r0r0r1describe_categoricalsz!PandasColumn.describe_categoricalcCst|jjtr tj}d}||fSt|jjtr^|jjjj d ddurTtj dfStj dfS|jd}zt |\}}Wn"tytd|dYn0||fS)NrrBz not yet supported)r&r-r:r r USE_BYTEMASKr array _pa_arraychunksbuffers NON_NULLABLEZ USE_BITMASK_NULL_DESCRIPTIONKeyErrorr+)r/Zcolumn_null_dtypeZ null_valuerEnullvaluer0r0r1 describe_nulls    zPandasColumn.describe_nullcCs|jS)zB Number of null elements. Should always be known. )r-Zisnasumitemr5r0r0r1 null_countszPandasColumn.null_countzdict[str, pd.Index]cCs d|jjiS)z8 Store specific metadata of the column. z pandas.index)r-indexr5r0r0r1metadataszPandasColumn.metadatacCsdS)zE Return the number of chunks the column consists of. rOr0r5r0r0r1 num_chunksszPandasColumn.num_chunksNz int | None)n_chunksccsr|rh|dkrht|j}||}||dkr2|d7}td|||D]"}t|jj||||jVqBn|VdS)zy Return an iterator yielding the chunks. See `DataFrame.get_chunks` for details on ``n_chunks``. rOrN)lenr-rangerZilocr.)r/rar4stepstartr0r0r1 get_chunkss    zPandasColumn.get_chunksrcCs\|ddd}z||d<Wnty2Yn0z||d<WntyVYn0|S)a` Return a dictionary containing the underlying buffers. The returned dictionary has the following contents: - "data": a two-element tuple whose first element is a buffer containing the data and whose second element is the data buffer's associated dtype. - "validity": a two-element tuple whose first element is a buffer containing mask values indicating missing data and whose second element is the mask value buffer's associated dtype. None if the null representation is not a bit or byte mask. - "offsets": a two-element tuple whose first element is a buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and whose second element is the offsets buffer's associated dtype. None if the data buffer does not have an associated offsets buffer. N)datavalidityoffsetsrhri)_get_data_buffer_get_validity_bufferr_get_offsets_buffer)r/rTr0r0r1 get_buffers s  zPandasColumn.get_buffersz.tuple[Buffer, tuple[DtypeKind, int, str, str]]c Cs|jdtjtjtjtjtjfvr|j}|jdtjkr^t|jddkr^|jj d }n^|jj }t |jjtr||j}n@t |jjtr|jjd}t|dt|d}||fS|j}t||jd}n|jdtjkr|jjj}t||jd}||j}nz|jdtjkrj|j }t}|D]$}t |tr*||j dd q*tt!j"|d d }|j}nt#d |jjd ||fS)zZ Return the buffer containing the data and the buffer's associated dtype. rNrOlength)r#utf-8encodingZuint8)r:rBr%)$r:rINTUINTFLOATrJDATETIMErbr-dtZ tz_convertto_numpyrQr&r _datar rRrSrrTZ_ndarrayr r.r>r;_codesr=r@ bytearraystrextendencodenpZ frombufferr+) r/r:Znp_arrarrbufferr<bufrobjr0r0r1rj0sH "     zPandasColumn._get_data_bufferztuple[Buffer, Any] | Nonec Csp|j\}}t|jjtrn|jjjjd}tj dt j t j f}| ddurNdSt| dt|d}||fSt|jjtr|jjj}t|}tj dt j t j f}||fS|jdtjkr4|j}|dk}| }tjt|ftjd}t|D] \} } t| tr|n||| <qt|}tj dt j t j f}||fSzt|d} WntybtdYn0t| dS) z Return the buffer containing the mask values indicating missing data and the buffer's associated dtype. Raises NoBufferPresent if null representation is not a bit or byte mask. rrONrpr9shaper:z! so does not have a separate maskzSee self.describe_null)rZr&r-r:r rQrRrSrrJrrr?rTrrbr Z_maskr r@rzrzerosZbool_ enumerater~_NO_VALIDITY_BUFFERrWr+r) r/rXinvalidrr:rmaskrZvalidrrmsgr0r0r1rkns>    z!PandasColumn._get_validity_bufferztuple[PandasBuffer, Any]c Cs|jdtjkr|j}d}tjt|dftjd}t |D]6\}}t |t rj|j dd}|t|7}|||d<q@t |}tjdtjtjf}ntd||fS)a Return the buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and the buffer's associated dtype. Raises NoBufferPresent if the data buffer does not have an associated offsets buffer. rrOrrrrs@zJThis column has a fixed-length dtype so it does not have an offsets buffer)r:rr@r-rzrrrbZint64rr&r~rr rurZINT64rr?r) r/r;Zptrrirvrrr:r0r0r1rls&    z PandasColumn._get_offsets_buffer)T)N)__name__ __module__ __qualname____doc__r2r4propertyr6rr:r=rNrZr]r_r`rfrmrjrkrlr0r0r0r1rHs, !  %>9r)3 __future__rtypingrrnumpyrZpandas._libs.librZpandas._libs.tslibsrZ pandas.errorsrZpandas.util._decoratorsrZpandas.core.dtypes.dtypesr Zpandasr'r r Zpandas.api.typesr Zpandas.core.interchange.bufferr rZ*pandas.core.interchange.dataframe_protocolrrrrZpandas.core.interchange.utilsrrrrrurvrwrJr@rxrCZUSE_NANZ USE_SENTINELrUr>rPrVrrr0r0r0r1sH