datacompy
DataComPy is a Python package designed for sophisticated comparisons of Pandas and Spark DataFrames. It extends basic equality checks by allowing the printing of statistics and customization of match accuracy. Initially an alternative to SAS's PROC COMPARE, it now supports multiple backends like Dask and Snowflake through simple integration options, enhancing performance across various platforms. The introduction of SparkSQLCompare further improves performance and aligns with ongoing updates in feature support and compatibility.
