Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on May 31, 2026

• [SPARK-55886][SQL][PYTHON] Add DataFrame.zip for merging column-projected DataFrames

  Show description for 177aee0
  

  zhengruifeng

  and

  cloud-fan

  committed
  177aee0

  Copy full SHA for 177aee0

Commits on May 27, 2026

• [SPARK-57072][PYTHON][DOC] Add missing 4.2 methods to PySpark API reference

  Show description for 64a8b51
  zhengruifeng

  committed
  64a8b51

  Copy full SHA for 64a8b51

• [SPARK-57070][PYTHON][DOC] Add `time_(from|to)_*` functions to Python API reference

  Show description for 04ce453
  zhengruifeng

  committed
  04ce453

  Copy full SHA for 04ce453

Commits on May 19, 2026

• [SPARK-56928][PYTHON] Increase minimum numpy version to 1.23.2

  Show description for 51ff9a1
  

  gaogaotiantian

  authored and

  dongjoon-hyun

  committed
  51ff9a1

  Copy full SHA for 51ff9a1

Commits on May 16, 2026

• [SPARK-56889][PYTHON][INFRA] Drop Python 3.10 Support

  Show description for 0a0d31b
  dongjoon-hyun

  committed
  0a0d31b

  Copy full SHA for 0a0d31b

Commits on May 15, 2026

• [SPARK-56820][SQL] Add counter_diff window function for converting cumulative counters to delta format

  Show description for 4b9b35b
  

  pnikic-db

  authored and

  cloud-fan

  committed
  4b9b35b

  Copy full SHA for 4b9b35b

Commits on May 14, 2026

• [SPARK-56854][PYTHON] Filter None values in DataFrame[Stream]Reader/Writer .option(s)

  Show description for f1cdb6d
  

  lavanv11

  authored and

  HyukjinKwon

  committed
  f1cdb6d

  Copy full SHA for f1cdb6d

• [SPARK-56395][CONNECT][PYTHON] Add NEAREST BY DataFrame API

  Show description for 13380e7
  

  dilipbiswal

  authored and

  cloud-fan

  committed
  13380e7

  Copy full SHA for 13380e7

Commits on May 10, 2026

• [SPARK-40336][PS] Implement DataFrameGroupBy.cov

  Show description for 7041d1e
  

  devin-petersohn

  authored and

  HyukjinKwon

  committed
  7041d1e

  Copy full SHA for 7041d1e

• [SPARK-40373][PS] Implement ps.show_versions

  Show description for 35f02b1
  

  devin-petersohn

  authored and

  HyukjinKwon

  committed
  35f02b1

  Copy full SHA for 35f02b1

Commits on May 6, 2026

• [SPARK-40341][PYTHON] Implement Rolling.median and Expanding.median in pandas API on Spark

  Show description for 00d2487
  

  devin-petersohn

  authored and

  HyukjinKwon

  committed
  00d2487

  Copy full SHA for 00d2487

• [SPARK-40337][PS] Implement SeriesGroupBy.describe in pandas API on Spark

  Show description for 11d51c7
  

  devin-petersohn

  authored and

  HyukjinKwon

  committed
  11d51c7

  Copy full SHA for 11d51c7

Commits on May 5, 2026

• [SPARK-52857] Add is_valid_variant expression

  Show description for 109a130
  

  chenhao-db

  authored and

  cloud-fan

  committed
  109a130

  Copy full SHA for 109a130

Commits on May 1, 2026

• [SPARK-56594][SQL] Add time_bucket scalar function

  Show description for 586ac79
  

  vranes

  authored and

  cloud-fan

  committed
  586ac79

  Copy full SHA for 586ac79

Commits on Apr 17, 2026

• [SPARK-56256][PYTHON][DOCS][FOLLOWUP] Document SparkSession.emptyDataFrame in API reference

  Show description for 4fab58a
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  4fab58a

  Copy full SHA for 4fab58a

• [SPARK-56518][PYTHON] Add current_path to PySpark functions

  Show description for 8312182
  

  srielau

  authored and

  HyukjinKwon

  committed
  8312182

  Copy full SHA for 8312182

Commits on Apr 15, 2026

• [SPARK-56351][PYTHON][DOCS] Add dedicated documentation page for Arrow Python UDFs

  Show description for da37644
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  da37644

  Copy full SHA for da37644

Commits on Apr 8, 2026

• [SPARK-56221][SQL][PYTHON][FOLLOWUP] Rename TablePartition and remove SHOW CACHED TABLES / listCachedTables

  Show description for a3930d3
  cloud-fan

  committed
  a3930d3

  Copy full SHA for a3930d3

Commits on Apr 7, 2026

• [SPARK-55450][SS][PYTHON][DOCS] Document admission control in PySpark streaming data sources

  Show description for 98cdaee
  

  jiteshsoni

  authored and

  HeartSaVioR

  committed
  98cdaee

  Copy full SHA for 98cdaee

Commits on Apr 3, 2026

• [SPARK-56341][PYTHON][DOCS] Fix outdated PyArrow minimum version in arrow_pandas.rst

  Show description for cba5705
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  cba5705

  Copy full SHA for cba5705

Commits on Mar 26, 2026

• [SPARK-56221][SQL][PYTHON] Feature parity between spark.catalog.* vs DDL commands

  Show description for 91d62ce
  HyukjinKwon

  committed
  91d62ce

  Copy full SHA for 91d62ce

Commits on Mar 23, 2026

• [SPARK-56135][DOCS] Add option to use venv in PySpark docs

  Show description for a5a7588
  

  Yicong-Huang

  authored and

  zhengruifeng

  committed
  a5a7588

  Copy full SHA for a5a7588

Commits on Mar 21, 2026

• [SPARK-55806][DOCS] add section for running `pyspark` with `uv` inline dependencies

  Show description for eab938b
  

  markjm

  authored and

  HyukjinKwon

  committed
  eab938b

  Copy full SHA for eab938b

Commits on Feb 23, 2026

• [SPARK-55558][SQL] Add support for Tuple/Theta set operations

  Show description for 4abee40
  

  2 peopleauthored and

  dtenedor

  committed
  4abee40

  Copy full SHA for 4abee40

Commits on Feb 19, 2026

• [SPARK-55540][DOCS] Fix capitalization inconsistency in PySpark Overview "Live Notebook" link

  Show description for a5f064f
  

  knkarthik01

  authored and

  HyukjinKwon

  committed
  a5f064f

  Copy full SHA for a5f064f

Commits on Feb 17, 2026

• [SPARK-55279][SQL][FOLLOW-UP] Add sketch_funcs and Datasketch grouping for SQL functions

  Show description for eb2c6f0
  

  2 peopleauthored and

  dtenedor

  committed
  eb2c6f0

  Copy full SHA for eb2c6f0

Commits on Feb 13, 2026

• [SPARK-55416][SS][PYTHON] Streaming Python Data Source memory leak when end-offset is not updated

  Show description for 7542473
  

  vinodkc

  authored and

  HeartSaVioR

  committed
  7542473

  Copy full SHA for 7542473

Commits on Feb 11, 2026

• [SPARK-54179][SQL][FOLLOW-UP] Add Dataframe API support for Tuple sketches

  Show description for 7798054
  

  2 peopleauthored and

  dtenedor

  committed
  7798054

  Copy full SHA for 7798054

Commits on Feb 5, 2026

• [SPARK-55086][PYTHON] Add DataSourceReader.pushFilters to Python Data Source API docs

  Show description for 508130f
  

  yamayuki-hub

  authored and

  HyukjinKwon

  committed
  508130f

  Copy full SHA for 508130f

Commits on Jan 28, 2026

• [SPARK-55179][PYTHON][CONNECT] Skip eager column name validation in `df.col_name`

  Show description for 4352988
  zhengruifeng

  committed
  4352988

  Copy full SHA for 4352988

Commits on Jan 20, 2026

• [SPARK-54849][PYTHON][FOLLOW-UP] Update minimum version of pyarrow in `setup.py` and `pyspark_upgrade.rst`

  Show description for 4d3b45d
  zhengruifeng

  committed
  4d3b45d

  Copy full SHA for 4d3b45d

Commits on Jan 12, 2026

• [SPARK-54785][SQL] Add support for binary sketch aggregations in KLL

  Show description for fc15f72
  

  2 peopleauthored and

  dtenedor

  committed
  fc15f72

  Copy full SHA for fc15f72

Commits on Jan 9, 2026

• [SPARK-53882][CONNECT][DOCS] Add documentation comparing behavioral differences between Spark Connect and Spark Classic

  Show description for 5e18c9d
  

  xi-db

  authored and

  hvanhovell

  committed
  5e18c9d

  Copy full SHA for 5e18c9d

Commits on Jan 4, 2026

• [SPARK-54543][PYTHON][INFRA] Retire flake8

  Show description for ea2617a
  

  gaogaotiantian

  authored and

  zhengruifeng

  committed
  ea2617a

  Copy full SHA for ea2617a

Commits on Dec 30, 2025

• [SPARK-54849][PYTHON] Upgrade the minimum version of `pyarrow` to 18.0.0

  Show description for f80b919
  zhengruifeng

  committed
  f80b919

  Copy full SHA for f80b919