About Kinetica
Kinetica is blazing fast on a GPU but even on CPUs we continue to outperform leading analytical databases in TPC-DS benchmarks. Comparisons are made to Kinetica from a geomean of the queries that a database was able to run. So Clickhouse was 13x slower overall on the 8 queries it was able to successfully complete. An independently designed and performed Spatial and Time Series Benchmark to help organizations evaluate database technologies suitable for IoT and sensor data workloads. This report evaluates the performance and functionality of leading cloud databases with built in geospatial, temporal, and graph capabilities Tests were run with 200GB (SF200) of sample data comprising 24 tables in a snowflake schema. The tables store web, catalog and store sales from an imaginary retailer. The largest fact table had well over a billion rows. Benchmarking was done using a consistent distributed hardware configuration of four Azure virtual machines: E48s v4 (48 vCPU, 384 GB RAM) with 2TB premium SSD, or equivalent setup. Kinetica’s unique native vectorized join engine is able to process chunks of data in parallel, rather than sequentially working through rows of data line by line. This delivers very quick results, particularly with complex ad-hoc analysis of complex data. Kinetica has been in development for over a decade. It is mature and battle-tested and able to reliably parse and deliver results on complex SQL queries. Kinetica is typically able to run all 99 queries of the TPC-DC harness. Kinetica fast, real-time capabilities extend out to it’s versatile
Key features of the Kinetica that are enabled through these drivers include:
- Python UDF API - Kinetica Docs
- Geospatial/Geometry Functions - Kinetica Docs
- SQL Developer Guide - Kinetica Docs
- Data Loading Concepts - Kinetica Docs
- The AI Database – A Prerequisite to Operationalizing - kinetica
- Postgis vs Geomesa vs Kinetica
- Java Developer Guide - Kinetica Docs
The following is a complete example, using the Python UDF API, of a non-CUDAUDF that demonstrates how to create pandas dataframes and insert them intotables in Kinetica. This example (andothers) can be found in the Python UDF APIrepository; this repository can be downloaded/cloned fromGitHub.ReferencesPython UDF Reference -- detaileddescription of the entire UDF APIRunning UDFs -- detailed description onrunning Python UDFsExample UDFs -- example UDFs written inPythonPrerequisitesThe general prerequisites for using UDFs in Kinetica can be found underUDF Prerequisites.ImportantThis example cannot run on Mac OSXThe following items are also necessary:Python 3MinicondaNoteVisit the Conda website to downloadthe Miniconda installer for Python 3.UDF API DownloadThis example requires local access to the Python UDF API repository. In thedesired directory, run the following but be sure to replace with the name of the installed Kinetica version, e.g,v7.2:git clone -b release/ --single-branch ScriptsThere are four files associated with the pandas UDF example, allof which can be found in the Python UDF API repo.A database setup script (test_environment.py) that is called from theinitialization scriptAn initialization script (setup_db.py) that creates the output tableA script (register_execute_UDF.py) to register and execute the UDFA UDF (df_to_output_UDF.py) that creates a pandas dataframe and insertsit into the output tableThis example runs inside a Conda environment. The environment can beautomatically configured using the conda_env_py3.yml file found in thePython UDF API repository.In the same directory you cloned the API, change directory into the rootfolder of the Python UDF API repository:cd kinetica-udf-api-python/Create the Conda environment, replacing with thedesired name:conda env create --name --file conda_env_py3.ymlNoteIt may take a few minutes to create the environment.Verify the environment was created properly:conda info --envsActivate the new environment:conda activate Install PyGDF:conda install -c numba -c conda-forge -c gpuopenanalytics/label/dev -c defaults pygdf=0.1.0a2Install the Kinetica Python API:pip install gpudb~=7.2.0Add the Python UDF API repo's root directory to the PYTHONPATH:export PYTHONPATH=$(pwd):$PYTHONPATHEdit the util/test_environment.py script for the correct database url,user, and password for your Kinetica instance:URL = ' = 'admin'PASSWORD = 'admin123'UDF DeploymentChange directory into the UDF Pandas directory:cd examples/UDF_pandasRun the UDF initialization script:python setup_db.pyRun the execute script for the training UDF:python register_execute_UDF.py --url --username --password Verify the results in GAdmin, either on thelogging page or in the unittest_df_outputoutput table on the table view page.Execution DetailThis example details using a distributed UDF to create and ingest a pandasdataframe into Kinetica. The df_to_output_UDF proc creates the dataframeand inserts it into the output table, unittest_df_output.The dataframe has a shape of (3, 3) and will get inserted into theoutput table n number of times, where n is equal to the number ofprocessing nodes available in each processing node container registered inKinetica.The output table contains 3 columns:id -- an integer columnvalue_long -- a long columnvalue_float -- a float columnDatabase SetupThe setup script, setup_db.py, which creates the output table for the UDF,imports the test_environment.py script to access its methods:1from util import test_environment as teThe script calls two methods from test_environment.py: one to create theschema used to contain the example tables and one to create the output tableswhere the values for the output table's name and type are passed in:12te.create_schema()te.create_test_output_table(te.TEST_OUTPUT_TABLE_NAME, te.TEST_DATA_TYPE_1)The
