-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
Description
Materializing table transformations helps improving runtime performance if the transformations are to be execute multiple times.
Some transformations on a table also change table schema based on content in the table.
When to materialize
- When user observe the table: We already compute all the transformations that builds up the table. We should materialize it, unless the user only observe a part of the table. Ex:
table.print({limit: 2}) - When user say so: We should allow users to control when a table can materialized on demand.
- When transforming with a verb that requires the knowledge of the data countent: For example,
pivotneeds to know all the possible values of the pivoting key column before calculating the schema of the output table.
Reactions are currently unavailable