weld

Using and developing Weld, a runtime for data analytics

Channels

Chat

# All channels

# General

Team

Shoumik Palkar

@shoumik

Posts

Chat

Members

Info

KueenLau

·7d# General

Question of C API to input a mutable struct

I have seen the demo in vector_benchmark.cpp. The input data is a fixed struct. My question is that how can I input the data whose structure cannot be pre-defined. For instance, different table has different schema.

heisenberg

·10d# General

An active weld community

I am very interested in this project and have already forked this project. Since the previous community is no longer active, I will work on the new weld community in the future. I have a preliminary understanding of this project, but I may not understand it correctly. I welcome…

KueenLau

·5d# General

An IR which returns a wrong answer.

I construct a test which uses a vector containing struct. But the length of result is 0. When I change the input data to i32 vector, the length of result is correct. Blow is the code. //Multi Column Test let code = "|v: vec[{i32,i32,i32}] | …

heisenberg

·19d# General

So many questions.

When I checked the benchmark tests of the project, I found that none of these tests seemed to be available. I don't know if you can use it. After modifying some parts, I found that under the Crime_Index test case, the performance did not improve but decreased. It seems that you…

Quino Terrasa

·19d# General

Is the project still active?

Hello everyone, I am very impressed about the work done in Weld, and I am even more thrilled about the usage of Rust / LLVM to optimize big data workloads. Still, I am willing to contribute or extend the project, but it seems the project is in a hiatus since April 2020. Have…

heisenberg

·19d# General

Grizzly performance degradation problem

I don’t know if you have tested this example: https://github.com/weld-project/weld/tree/master/examples/python/grizzly/get_population_stats_grizzly.py and get_population_stats_simplified_grizzly.py. In these two examples, performance has not improved. Is the method I tested…

Adam Mustafa

·33d# General

Activity?

Hello all, I just stumbled upon weld and am very excited about its potential. There doesn't seem to have been any activity for a while (since April). I imagine Covid-19 may have had something to do with it but is the project still active?

evanrichter

·362d# General

Documentation for creating a new hardware backend?

As I see it the possible backends are LLVM (x86 supported) or openCL (for gpus). I've been tinkering with a (now deprecated) xeonphi coprocessor, with the knc (knight's corner) instruction set. LLVM doesn't support this family of x86 because of it's strange 512-bit pipeline…

Mossa Merhi Reimert

·1y# General

Rust, weld, R and parallelism

I've watched the Weld-talk online, and I am partially hooked. Recently, I've spent a lot of time and effort into learning rust, and I am thus far impressed with it as a programming language, but I wish to understand the landscape of scientific computing a bit more. What are the…

Christopher Andronikos

·1y# General

Runtime struct packing to byte array and typecasting slices

Is it possible to generalise the input data to accept a byte array of a manually packed struct and typecast individual slices back to original type from weld?