basics-syntax-comparision

# 1. SELECT
SELECT emp_id, emp_name, salary FROM employees;

df.select("emp_id", "emp_name", "salary")
--------------------------------------------------
# 2. WHERE / FILTER
SELECT * FROM employees WHERE salary > 50000;

from pyspark.sql.functions import col
df.filter(col("salary") > 50000)
-------------------------------------------------
# 3. Multiple Conditions
SELECT * FROM employees WHERE salary > 50000 AND age > 25;

df.filter((col("salary") > 50000) & (col("age") > 25))

Interview Question:
Why use & and not and?
-> and is Python logical operator
-> & works on Spark Column objects
-------------------------------------------------
# 4. ORDER BY
SELECT * FROM employees ORDER BY salary DESC;

df.orderBy(col("salary").desc())
--------------------------------------------------
#5. LIMIT
SELECT * FROM employees LIMIT 5;

df.limit(5)
--------------------------------------------------