listagg

Aggregate function: returns the concatenation of non-null input values, separated by the delimiter.

For the corresponding Databricks SQL function, see listagg aggregate function.

Syntax

import pyspark.sql.functions as sf

sf.listagg(col=<col>)

# With delimiter
sf.listagg(col=<col>, delimiter=<delimiter>)

Parameters

Returns

pyspark.sql.Column: the column for computed results.

Examples

Example 1: Using listagg function.

import pyspark.sql.functions as sf
df = spark.createDataFrame([('a',), ('b',), (None,), ('c',)], ['strings'])
df.select(sf.listagg('strings')).show()

+----------------------+
|listagg(strings, NULL)|
+----------------------+
|                   abc|
+----------------------+

Example 2: Using listagg function with a delimiter.

import pyspark.sql.functions as sf
df = spark.createDataFrame([('a',), ('b',), (None,), ('c',)], ['strings'])
df.select(sf.listagg('strings', ', ')).show()

+--------------------+
|listagg(strings, , )|
+--------------------+
|             a, b, c|
+--------------------+

Example 3: Using listagg function with a binary column and delimiter.

import pyspark.sql.functions as sf
df = spark.createDataFrame([(b'\x01',), (b'\x02',), (None,), (b'\x03',)], ['bytes'])
df.select(sf.listagg('bytes', b'\x42')).show()

+---------------------+
|listagg(bytes, X'42')|
+---------------------+
|     [01 42 02 42 03]|
+---------------------+

Example 4: Using listagg function on a column with all None values.

import pyspark.sql.functions as sf
from pyspark.sql.types import StructType, StructField, StringType
schema = StructType([StructField("strings", StringType(), True)])
df = spark.createDataFrame([(None,), (None,), (None,), (None,)], schema=schema)
df.select(sf.listagg('strings')).show()

+----------------------+
|listagg(strings, NULL)|
+----------------------+
|                  NULL|
+----------------------+