WEBVTT 00:00:00.000 --> 00:00:02.970 Strata is a specialized inference engine for one model, 00:00:02.970 --> 00:00:04.938 Qwen 3.8 Flash Next. 00:00:04.972 --> 00:00:07.941 The model itself is incredible, and Strata has really 00:00:08.976 --> 00:00:09.943 amazing performance. 00:00:09.977 --> 00:00:13.947 I'm getting almost 100 tokens per second on a 177 billion 00:00:13.981 --> 00:00:17.951 parameter model on my 4090 at home, which I didn't think was possible. 00:00:17.985 --> 00:00:19.953 The model has incredible caching behavior. 00:00:20.988 --> 00:00:23.957 It anticipates cache misses and starts streaming 00:00:23.991 --> 00:00:26.960 them to the GPU ahead of time, and it uses all the 00:00:26.994 --> 00:00:28.962 resources on your machine to do inference super fast. 00:00:29.997 --> 00:00:30.964 That all sounds amazing. 00:00:30.998 --> 00:00:32.966 I won't be using Strata. 00:00:33.000 --> 00:00:34.968 I don't recommend using it. 00:00:34.968 --> 00:00:37.938 My preference in my deployment model is for local only 00:00:37.971 --> 00:00:39.940 offline inference. 00:00:39.973 --> 00:00:42.943 Strata doesn't fit with that model outline of this video. 00:00:43.977 --> 00:00:45.946 I'm going to put the cliff notes at the very front. 00:00:45.979 --> 00:00:47.948 My recommendation is don't install it. 00:00:47.981 --> 00:00:50.951 that YOLOs random blobs from a 2 week old repo. 00:00:51.985 --> 00:00:53.954 Second, the model itself is incredible. 00:00:53.987 --> 00:00:54.955 It's a new model from Qwen. 00:00:56.990 --> 00:00:59.960 Third Strata and how it works, this is the meat of the video. 00:00:59.993 --> 00:01:03.964 My conclusion is that there are 2 main tricks, parallelism and caching. 00:01:03.997 --> 00:01:05.966 We're going to visualize the results of this. 00:01:06.967 --> 00:01:08.936 Fourth, what does this mean for llama.cpp? 00:01:09.970 --> 00:01:12.940 If this is a race, the GGML guys are running backwards. 00:01:12.973 --> 00:01:15.943 And I think it's extremely relevant to discussing Strata. 00:01:15.976 --> 00:01:18.946 And lastly, my fork of Strata for safe 00:01:18.979 --> 00:01:20.948 offline use. 00:01:20.981 --> 00:01:23.951 So my recommendation in a little bit more 00:01:23.984 --> 00:01:26.954 detail is do not use Strata, or use docker 00:01:27.988 --> 00:01:31.959 to run it, or consider wiping your machine after you install it and run it. 00:01:32.993 --> 00:01:35.963 Inference servers are probably the most critical 00:01:35.996 --> 00:01:38.966 pieces of infrastructure that we have in the home lab community 00:01:38.999 --> 00:01:41.969 or folks who are doing inference in, you know, 00:01:41.969 --> 00:01:43.937 at home or at work. 00:01:43.971 --> 00:01:46.940 And I don't think that they should have broad permissions and 00:01:46.974 --> 00:01:48.942 persistent access to the greater internet. 00:01:48.976 --> 00:01:51.945 So, Strata is violating at least 3 cardinal sins 00:01:51.979 --> 00:01:52.946 of software for these machines. 00:01:54.982 --> 00:01:57.951 One is downloading random binaries and running them from a two-week 00:01:57.985 --> 00:01:58.952 old GitHub repo. 00:01:58.986 --> 00:01:59.953 It's just not a good idea. 00:01:59.987 --> 00:02:01.955 I don't care if it's Strata or something else. 00:02:01.989 --> 00:02:02.956 I don't recommend this. 00:02:03.991 --> 00:02:06.960 Second, is installation by downloading scripts 00:02:06.994 --> 00:02:09.963 and markdown and yeeting that shit into your agent of choice. 00:02:09.997 --> 00:02:11.965 Also not recommended. 00:02:11.999 --> 00:02:13.967 2026 is a low trust environment, all right? 00:02:15.969 --> 00:02:18.939 This is like piping curl into bash on steroids 00:02:18.972 --> 00:02:20.941 and crack and a huge dose of creatine. 00:02:20.974 --> 00:02:23.944 Just, no, it's the mother of all attack vectors. 00:02:23.977 --> 00:02:25.946 The thing has 17,000 GitHub stars. 00:02:25.979 --> 00:02:28.949 It could have 10 X that and the number of installs. 00:02:28.982 --> 00:02:30.951 It's a very juicy attack target, okay? 00:02:31.985 --> 00:02:35.956 So even if it's not compromised today, treat it with suspicion. 00:02:36.990 --> 00:02:38.959 And guys, this is not specific to Strata. 00:02:39.993 --> 00:02:41.962 I felt the same way about OpenClaw. 00:02:41.995 --> 00:02:44.965 Okay, the 3rd cardinal sin is Strata's setup.py, 00:02:45.999 --> 00:02:48.969 download shit automatically from GitHub without asking you, maybe not 00:02:48.969 --> 00:02:50.937 malicious, but absolutely not cool. 00:02:50.937 --> 00:02:52.939 So, uh, Niko, I chatted 00:02:53.974 --> 00:02:56.977 with the gentleman who appears to be in control of the GitHub repo, 00:02:56.977 --> 00:02:59.946 and he seems like a very nice guy, and he said that he would address these issues. 00:03:01.982 --> 00:03:04.951 So in general, like be skeptical, okay? 00:03:04.985 --> 00:03:06.953 At the simplest end, use a container. 00:03:06.987 --> 00:03:09.956 I don't see enough people using common sense about this 00:03:09.990 --> 00:03:12.959 kind of stuff, like OpenClaw, dots, whatever 00:03:12.993 --> 00:03:15.962 metas, free AI agent is called. 00:03:16.997 --> 00:03:18.965 This is why I love llama.cpp. 00:03:18.999 --> 00:03:20.967 I download the source code. 00:03:20.967 --> 00:03:22.969 I rsync it to my inference machine, and then I run make. 00:03:23.970 --> 00:03:24.938 Done. 00:03:24.971 --> 00:03:26.940 Can you guys imagine how fun I am at parties? 00:03:27.974 --> 00:03:30.944 I'm the guy who shows up and then I'm going through your cupboard for the spice 00:03:30.977 --> 00:03:32.946 cabinet and checking the expiration date on everything. 00:03:33.980 --> 00:03:34.981 Base model intelligence. 00:03:34.981 --> 00:03:35.949 It's incredible. 00:03:35.982 --> 00:03:37.951 It made this beautiful pagoda. 00:03:37.984 --> 00:03:40.954 It made this super Dario platformer where Dario 00:03:40.987 --> 00:03:43.957 runs around and turns all the lakes into money. 00:03:44.991 --> 00:03:45.992 We'll do a full video on that. 00:03:45.992 --> 00:03:46.960 It made this graphic calculator. 00:03:47.994 --> 00:03:48.962 It's really smart. 00:03:49.996 --> 00:03:52.966 But like I said earlier, it's a 177B parameter 00:03:52.999 --> 00:03:53.967 model from Qwen, it's going to slap. 00:03:55.969 --> 00:03:56.937 Okay, Strata. 00:03:56.970 --> 00:03:59.940 Yeah, the main part of this video is about Strata, how it works, how it's so fast. 00:04:00.974 --> 00:04:04.945 I got 90 tokens per second and 2500 prompt processing. 00:04:04.978 --> 00:04:06.947 It does go down as you fill the context window. 00:04:06.980 --> 00:04:09.950 That's like 2 to 3 times as fast as llama.cpp 00:04:09.983 --> 00:04:11.952 on the same model with essentially the same settings. 00:04:12.986 --> 00:04:13.954 There's one catch here. 00:04:13.987 --> 00:04:17.958 I ran it on a RunPod container and on a container on my machine at home. 00:04:18.992 --> 00:04:21.962 And the physical machine had 128 cores 00:04:21.995 --> 00:04:25.966 and 256 threads, and I had like 32 00:04:25.999 --> 00:04:28.969 of those for my, uh, container 00:04:28.969 --> 00:04:30.937 that was allocated to me on RunPod. 00:04:31.972 --> 00:04:34.941 And Strata spawned way too many loader threads and 00:04:34.975 --> 00:04:35.942 nuked its own performance. 00:04:35.976 --> 00:04:38.945 So check the num_workers flag and that can 00:04:40.981 --> 00:04:42.983 severely hamper your day if you're trying to run. 00:04:42.983 --> 00:04:44.951 My agent found it really quickly. 00:04:44.985 --> 00:04:45.952 It wasn't a big deal. 00:04:45.986 --> 00:04:48.955 In terms of design, everything 00:04:48.989 --> 00:04:51.958 follows from this one idea, keep the GPU 00:04:51.992 --> 00:04:53.960 busy at all times during decode and prompt processing. 00:04:55.996 --> 00:04:59.966 This clip visualizes the performance data captured by NVIDIA 00:05:00.000 --> 00:05:02.969 Nsight, my agent, I believe it was Astra 00:05:02.969 --> 00:05:05.939 or Fable assembled the frames into Swift, adding 00:05:05.972 --> 00:05:08.942 some metadata to them, but the key tool 00:05:08.975 --> 00:05:10.944 here is NVIDIA Nsight. 00:05:10.977 --> 00:05:13.947 This is a real-time render of llama.cpp performing 00:05:14.981 --> 00:05:17.951 a pre-fill of 4096 bytes. 00:05:17.984 --> 00:05:20.954 The 1st of all, the function that's currently executing is being shown. 00:05:20.987 --> 00:05:23.957 So mul_mat_q is the matrix by matrix 00:05:23.990 --> 00:05:26.960 kernel in llama.cpp, and for a given 1/30th of a 00:05:26.993 --> 00:05:29.963 second window, which is each frame in the in the 00:05:29.996 --> 00:05:32.966 video, it shows the kernel that had the largest total 00:05:32.999 --> 00:05:35.969 overlap with the window, so not necessarily the longest kernel 00:05:36.970 --> 00:05:39.940 or the most recent one, but the one that had the most overlap. 00:05:40.974 --> 00:05:44.945 The blue teal seafoam green swim lane, the GPU being busy. 00:05:44.978 --> 00:05:45.946 We want to see that full. 00:05:45.979 --> 00:05:46.947 It's not full. 00:05:46.980 --> 00:05:49.950 If I had to give llama a grade on this metric, I 00:05:49.983 --> 00:05:51.952 would give it a D for didn't do it. 00:05:51.952 --> 00:05:54.955 The ambiguously orange yellow lane is moving memory 00:05:54.988 --> 00:05:58.959 over PCIE triggered by CUDA's cudaMemcpy. 00:05:59.993 --> 00:06:02.963 And then pink is the GPU being idle. 00:06:02.996 --> 00:06:03.964 this is constructed after the fact. 00:06:03.997 --> 00:06:06.967 Retrospectively, ex post facto. 00:06:07.000 --> 00:06:07.968 later, okay? 00:06:07.968 --> 00:06:09.936 The CPU is the white line. 00:06:09.970 --> 00:06:12.939 If it's full to the top, that's one core in use, and then the gray 00:06:12.973 --> 00:06:15.942 means more than one core of CPU is being used. 00:06:16.977 --> 00:06:18.945 So, guys, is llama doing a good job? 00:06:18.979 --> 00:06:20.947 Now, the next one is Strata. 00:06:20.981 --> 00:06:23.950 Look at that beautiful teal mass, and it's 00:06:23.984 --> 00:06:25.952 done in only 4.8 seconds. 00:06:26.987 --> 00:06:27.954 Gorgeous. 00:06:27.988 --> 00:06:30.957 I don't know why people are playing AION 2 when they can look at this. 00:06:31.992 --> 00:06:34.961 All right, so how does it do this? 00:06:34.995 --> 00:06:38.965 If you want, you can go watch Codacus's video on the topic. 00:06:40.000 --> 00:06:42.969 I watched it after inspecting the code and spending, I 00:06:42.969 --> 00:06:45.939 don't know, 5 or 6 hours between reading it, having my 00:06:45.972 --> 00:06:49.943 agents look at Strata and running both 00:06:49.976 --> 00:06:52.946 Strata and llama and llama forks that are purporting 00:06:52.979 --> 00:06:53.947 to do things similar to Strata. 00:06:54.981 --> 00:06:55.949 As I was watching the video, 00:06:56.983 --> 00:06:59.953 I could feel myself getting smarter in the areas 00:06:59.986 --> 00:07:02.956 of my brain, which are small but present and know things about back propagation. 00:07:03.990 --> 00:07:06.960 And I could feel my visual cortex physically 00:07:06.993 --> 00:07:09.963 decaying and getting stupider from watching these AI 00:07:09.996 --> 00:07:11.965 generated stick figures run around in a kitchen. 00:07:13.967 --> 00:07:16.936 So maybe just listen to the video from Codacus 00:07:16.970 --> 00:07:19.940 and don't watch the AI generated cooks pass things around in the pantry. 00:07:20.974 --> 00:07:24.944 My point is, he covers it in really great detail, and 00:07:24.978 --> 00:07:28.948 talks about a lot of the mechanisms within Strata, 00:07:28.982 --> 00:07:31.952 and I think the video is actually pretty accurate, according to 00:07:31.985 --> 00:07:33.954 my understanding of how it works. 00:07:34.988 --> 00:07:37.958 Now, to be clear, I talk 00:07:37.991 --> 00:07:40.960 about llama.cpp and running inference engines a lot. 00:07:40.994 --> 00:07:43.964 So asking me how they work is kind of like asking a truck driver 00:07:43.997 --> 00:07:44.964 to design a Mack truck. 00:07:45.999 --> 00:07:47.968 So what are the main tricks? 00:07:47.968 --> 00:07:49.936 Keep the hot experts in VRAM, okay? 00:07:50.970 --> 00:07:52.939 This is this is critical. 00:07:52.972 --> 00:07:55.942 Recalculate which ones are hot as you are going. 00:07:55.976 --> 00:07:57.944 It does not keep a simple LRU cache. 00:07:58.978 --> 00:08:02.949 It maintains a distribution of the experts that get used the most. 00:08:03.984 --> 00:08:07.954 Anticipate the upcoming cache misses and stream them into VRAM. 00:08:07.988 --> 00:08:10.957 This is going to allow us to see that ambiguously yellow orange region 00:08:11.992 --> 00:08:15.962 stay busy as it's constantly loading the 00:08:15.996 --> 00:08:17.964 next layer's experts that are going to get mixed. 00:08:18.998 --> 00:08:21.968 Now, it does a few interesting things that I view 00:08:22.969 --> 00:08:23.937 as corollaries to that. 00:08:23.970 --> 00:08:27.941 And one is that it has a separate binary pack file for the model. 00:08:27.974 --> 00:08:31.945 I thought this was really dumb at 1st because we have GGUFs, okay? 00:08:31.978 --> 00:08:35.949 And the GGUF contains everything that you need to run it. 00:08:35.982 --> 00:08:38.952 But it turns out that within a GGUF, 00:08:38.985 --> 00:08:41.955 a single expert is contained 00:08:41.988 --> 00:08:42.956 within 3 different tensors. 00:08:42.989 --> 00:08:46.960 And so you need to create the pack file, just to keep the 00:08:46.993 --> 00:08:51.965 experts in a format on disk, such that you can easily cache them. 00:08:51.998 --> 00:08:53.967 It actually makes a lot of sense. 00:08:53.967 --> 00:08:56.936 So like I said, everything follows from this one idea. 00:08:57.971 --> 00:08:59.939 Keep the GPU busy at all times. 00:08:59.939 --> 00:09:00.940 And llama.cpp just fails at this. 00:09:02.976 --> 00:09:05.979 So, on that note, like, 00:09:05.979 --> 00:09:08.948 this is why we should care about llama.cpp and talk about it here because 00:09:08.982 --> 00:09:12.952 I think that frameworks 00:09:12.986 --> 00:09:16.956 like ninfer, like Strata, like FreeToken. 00:09:17.991 --> 00:09:20.960 Oh, and DeepSeek or Dwarf Star, 00:09:20.994 --> 00:09:24.964 these are going to become more and more prevalent and important and 00:09:24.998 --> 00:09:27.967 be the choice that people reach for far more frequently. 00:09:27.967 --> 00:09:29.936 When it comes to new frameworks, 00:09:30.970 --> 00:09:32.939 I don't go off and review every single one. 00:09:32.972 --> 00:09:35.942 But that's why I'm spending the time here 00:09:35.975 --> 00:09:39.946 on Strata because this is an example of 00:09:39.979 --> 00:09:43.950 this new, very interesting class of specialized inference frameworks. 00:09:44.984 --> 00:09:47.954 And I think it's going to overtake llama.cpp and GGML 00:09:48.988 --> 00:09:50.957 if they don't change the way that they are operating. 00:09:51.991 --> 00:09:54.961 So, I predict that we're going to see more cumulative 00:09:55.995 --> 00:09:58.965 GitHub stars on things like ninfer and Strata than 00:09:58.998 --> 00:09:59.966 on llama.cpp. 00:10:00.967 --> 00:10:05.939 And one of the problems that they're having is they don't view contributions from the community. 00:10:06.973 --> 00:10:09.943 Update from the future, and it looks like all the attention to Strata got 00:10:09.976 --> 00:10:13.947 to them, and they started paying some attention to these PRs, 00:10:13.980 --> 00:10:17.951 and Georgi actually merged this GPU 00:10:17.984 --> 00:10:20.954 cache for MoE experts earlier this week on Wednesday. 00:10:21.988 --> 00:10:26.960 And one of the problems that they're having is they don't view contributions 00:10:26.993 --> 00:10:28.962 from the community, period. 00:10:28.995 --> 00:10:31.965 I looked at 15 different pull 00:10:32.999 --> 00:10:35.969 requests to llama.cpp for this MoE 00:10:35.969 --> 00:10:37.937 expert caching in VRAM. 00:10:38.972 --> 00:10:41.941 And I don't know if they're choosing not to merge them or not looking at them, 00:10:41.975 --> 00:10:43.943 but something's going wrong. 00:10:43.943 --> 00:10:45.945 And llama.cpp is so desperately 00:10:46.980 --> 00:10:49.983 behind on this, this one thing, execution 00:10:49.983 --> 00:10:51.951 of MoE efficiently. 00:10:52.986 --> 00:10:55.955 So, I don't want llama to lose relevance. 00:10:55.989 --> 00:10:58.958 I love the fact that I can clone it and copy it to my server and 00:10:58.992 --> 00:11:00.960 run make, and then I'm good to go. 00:11:01.995 --> 00:11:03.963 But it's not looking good for our heroes. 00:11:03.997 --> 00:11:06.966 So in addition, If llama 00:11:07.967 --> 00:11:10.937 were to lose relevance, blind 00:11:10.970 --> 00:11:13.940 trust in highly internet connected new frameworks, 00:11:13.973 --> 00:11:15.942 like Strata, it's definitely a bad thing. 00:11:17.977 --> 00:11:20.947 Okay, so my fork of Strata. 00:11:21.981 --> 00:11:24.951 I would rather have my daily driver Windows machine auto 00:11:24.984 --> 00:11:27.954 run a USB stick that I found in the parking lot that said 00:11:27.987 --> 00:11:30.957 Stuxnet on the side and Sharpie than install Strata 00:11:30.990 --> 00:11:32.959 on my home inference machine. 00:11:33.993 --> 00:11:34.961 I want to run it though. 00:11:34.994 --> 00:11:35.962 I want all the good stuff from it. 00:11:35.995 --> 00:11:38.965 So I took Strata, and I told my agents 00:11:38.998 --> 00:11:41.968 to remove all the Python, and all the downloading 00:11:41.968 --> 00:11:44.938 random shit at build time, and it 00:11:45.972 --> 00:11:47.941 works a lot like the llama.cpp model. 00:11:47.974 --> 00:11:49.943 So the tokenizer is now in C++. 00:11:50.977 --> 00:11:53.947 And the templating and the front-end 00:11:53.980 --> 00:11:56.950 server, basically all the Python junk got deleted 00:11:56.983 --> 00:11:58.952 and replaced with C++ by my agents. 00:11:58.985 --> 00:12:01.955 And it works pretty well, but 00:12:01.988 --> 00:12:04.958 I haven't really stress tested it to the extent that others have 00:12:04.991 --> 00:12:06.960 probably stress tested Strata. 00:12:07.994 --> 00:12:09.963 So, take it with a grain of salt. 00:12:09.996 --> 00:12:12.966 I think we should also be careful about influencers and content creators 00:12:12.999 --> 00:12:16.970 who are pitching us on products that they are using. 00:12:16.970 --> 00:12:18.938 Maybe they're vibe coding something and putting it out there. 00:12:19.973 --> 00:12:23.943 I don't feel great about that, but in this case, if 00:12:23.977 --> 00:12:26.946 you download this and use it the way it's intended, I will never know, 00:12:26.980 --> 00:12:29.949 because you'll clone it, you'll copy the 00:12:29.983 --> 00:12:33.953 tarball to a machine that's completely offline and then you'll run make. 00:12:34.988 --> 00:12:36.956 And I won't be involved. 00:12:37.991 --> 00:12:40.960 So, the fork requires no 00:12:40.994 --> 00:12:43.963 network access to build and it doesn't download random binaries. 00:12:43.997 --> 00:12:47.967 It does an auto update and replace stuff on disk without asking you. 00:12:48.968 --> 00:12:51.938 It's a simplified version of Strata, without 00:12:51.971 --> 00:12:53.940 any additional smarts or anything. 00:12:53.973 --> 00:12:56.943 So, get your CUDA drivers on your inference 00:12:56.976 --> 00:12:59.946 box, get the CDROM that came with your 5090, 00:12:59.979 --> 00:13:03.950 install that stuff, and then you'll be good to go. 00:13:03.983 --> 00:13:06.953 If you want to use it, you can, if you want to use Strata, you can also do that. 00:13:07.987 --> 00:13:09.956 It's a free country, but it used to be. 00:13:10.990 --> 00:13:13.960 In conclusion, I had this moment when I read the PageRank paper. 00:13:14.994 --> 00:13:18.965 And again, when I read the Chord distributed 00:13:18.998 --> 00:13:21.968 hash table paper, where I felt like 00:13:21.968 --> 00:13:24.937 this kind of experiencing a 00:13:24.971 --> 00:13:27.940 spark of genius, you know, where I was reading this paper and I was like, wow. 00:13:28.975 --> 00:13:31.944 That's amazing, and people are using this to actually build real products, 00:13:31.978 --> 00:13:34.947 like, you know, the massive databases 00:13:34.981 --> 00:13:37.950 behind Google and indexing the web. 00:13:37.984 --> 00:13:40.953 And then, so that's one way to experience something 00:13:41.988 --> 00:13:42.955 new technologically. 00:13:42.989 --> 00:13:44.957 The other way to do it. 00:13:44.991 --> 00:13:47.960 is the way Garry Kasparov experienced the 00:13:48.995 --> 00:13:49.962 genius of chess engines. 00:13:49.996 --> 00:13:53.966 And when he finally lost IBM's Deep Blue, he accused the team of cheating. 00:13:54.967 --> 00:13:57.937 And he thought that they had been secretly using Grandmasters. 00:13:58.971 --> 00:14:01.941 Because Garry saw deep genius in 00:14:01.974 --> 00:14:03.943 what the computer was doing for the 1st time. 00:14:04.977 --> 00:14:08.948 Now, today, if you're playing chess and they accuse you of cheating is because you're using a computer. 00:14:08.981 --> 00:14:11.951 not because you're using a grandmaster, but it was different back then. 00:14:11.984 --> 00:14:13.953 And so I don't know which one it is for me. 00:14:14.987 --> 00:14:17.957 Because when I sat down and I understood what Strata 00:14:17.990 --> 00:14:20.960 was doing and how it was working, I had that 00:14:20.993 --> 00:14:24.964 same moment of realization of like, wow, this is, this is incredible. 00:14:25.998 --> 00:14:28.968 It was especially highlighted to me when I saw how poorly 00:14:29.969 --> 00:14:31.938 llama.cpp does the same thing. 00:14:32.972 --> 00:14:35.942 Watching the traces, and seeing llama.cpp 00:14:35.975 --> 00:14:38.945 drop the ball on pre-fetching, and 00:14:38.978 --> 00:14:40.947 on keeping the GPU busy. 00:14:42.982 --> 00:14:44.951 I was like, wow, I get it. 00:14:45.985 --> 00:14:49.956 My opinion, is that somebody with deep technical 00:14:49.989 --> 00:14:52.992 expertise sat down for weeks planning and executing 00:14:52.992 --> 00:14:54.961 and designing Strata. 00:14:56.996 --> 00:15:00.967 And I would like to see the planning 00:15:00.967 --> 00:15:03.970 traces and conversations from it because they're not committed. 00:15:03.970 --> 00:15:06.939 It starts with the 1st commit is source code. 00:15:07.974 --> 00:15:10.943 So, I told this to Astra and 00:15:10.977 --> 00:15:12.945 I was like, wow, man, isn't this genius? 00:15:12.979 --> 00:15:16.949 I mean, they do all this pre-fetching and uh, 00:15:16.983 --> 00:15:18.951 they've got the VRAM cache. 00:15:18.985 --> 00:15:19.952 Like, this is great. 00:15:19.986 --> 00:15:20.953 And Astra was like, eh. 00:15:21.988 --> 00:15:25.958 They cite a couple papers and they're using some ideas from llama.cpp 00:15:25.992 --> 00:15:27.960 and they're pulling in some kernels. 00:15:27.994 --> 00:15:28.961 Isn't that great? 00:15:29.996 --> 00:15:30.963 I was like, fuck you, Astra. 00:15:31.998 --> 00:15:32.999 Okay, to wrap up. 00:15:32.999 --> 00:15:33.966 Strata is incredible. 00:15:34.967 --> 00:15:35.935 Don't use it. 00:15:35.968 --> 00:15:38.938 The guy who seems to control the GitHub repo chatted with me 00:15:38.971 --> 00:15:41.941 and agrees about the security issues. 00:15:41.974 --> 00:15:42.942 He says he's going to resolve some of them. 00:15:42.975 --> 00:15:45.945 If you enjoyed the video, like the video, 00:15:45.978 --> 00:15:48.948 consider subscribing to the channel, if you didn't like the video, press the down vote 00:15:48.981 --> 00:15:51.951 button, make sure you press it twice because it only works if you press it twice. 00:15:52.985 --> 00:15:55.955 I'm going to see you guys in the next one, but 1st I want to give a shout out to our 2 00:15:55.988 --> 00:15:57.957 YouTube members, zirconprime and Kruz_AI. 00:15:57.990 --> 00:16:00.960 Thank you guys for supporting the channel in this way, and I'll see you in the next one.