Miscellaneous Points

  • Databricks Workspace: Databricks platform that can integrate with CSP services
  • Databricks SQL Analytics: Allows to run SQL Queries on Data Lake
  • Hive metastore is used by Databricks by default to register databases, tables and views
  • Data from self-describing file formats can be read using SELECT * FROM file-format.'/path/to/file' (Can be used for others as well but the results might not be proper)
  • External Data Sources are not automatically updated it needs to be updated using the REFRESH command
  • The constraint that are applied on a table can be seen in the TLBPROPERTIES field of table description