Can't wait for the DwarfStar quants - I have been using DeepSeek v4 flash (preview) as my main coding agent for months now (running on my 128gb mbp) - it seems this model outperforms GLM 5.2 on nearly every metric. Thanks for sharing the news, I was refreshing huggingface but gave up thinking it likely would take some more time.
vmt-man 44 minutes ago [-]
are you working in earplugs? :)) even with 128 gigs of ram it must be super noisy.
kamranjon 9 minutes ago [-]
I actually run it as a server - so most of the time I don't have to listen to it right next to me - it's just sitting in another room in my house - but I often am traveling with it and will have it sitting right next to my coding laptop and yea the fan runs non-stop - it's not obnoxious so i can pretty easily tune it out - also airpods/noise canceling headphones help!
hetspookjee 25 minutes ago [-]
the noise is pretty doable, and so is the heat, I find. I read these messages before I had the machine and expected far worse. The tok/s is for me the dealbreaker as I prefer to have multiple sessions. For nightly runs i do like it a lot, or to be a node in a mesh.
theturtletalks 54 minutes ago [-]
What kind of tps are you getting?
kamranjon 16 minutes ago [-]
Generally get 20-25tps - prefill is pretty good around 400-450tps. I have been using compaction at around 100k tokens but mostly just cause it was the default in pi coding agent - might see if i can expand it a bit.
WithinReason 29 minutes ago [-]
Hmm, it's targeting a HW accelerator with a 128x128 matmul primitive, which one is that? Warp Group Matrix Multiply Accumulate on H100?
ttul 19 minutes ago [-]
Google TPUs are built around a 128×128 systolic array of multiply-accumulate (MAC) units. Trainium 1, Trainium 2, and Inferentia 2 also feature a 128x128 systolic array.
You learn something every day. Today, it was the term "systolic array": A systolic array is a specialized grid of simple, interconnected processing units designed to execute parallel data operations—like matrix multiplication—by rhythmically passing data directly from cell to neighboring cell without writing intermediate results back to main memory.
The term comes from the biological word systole (the contraction of the heart pumping blood through the body). In a systolic array, data "pulses" through a network of processing elements on every clock cycle, driven by a global clock beat.
andersa 8 minutes ago [-]
Just because scales are grouped by 128x128 tiles, does not mean you need a single compute tile that large. It works completely fine to process it with multiple smaller tiles that get given the same scales, like how this works on Hopper and Blackwell today
nolist_policy 14 minutes ago [-]
Huawei Ascend presumably?
baalimago 3 hours ago [-]
New Deepseek models are like Christmas for me. Really big fan of low cost API models, noone does it better than DS. Until VRAM price is low enough to run models locally, this is the way to go.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
dpacmittal 45 minutes ago [-]
Can't wait for China to catchup on hardware (memory and compute) and absolutely crush American companies in both price and performance.
baalimago 28 minutes ago [-]
Personally I'm hoping for EU to step up a bit
speedgoose 2 minutes ago [-]
Sorry we a focusing on leather handbags there.
tao_oat 9 minutes ago [-]
Seems incredibly unlikely, doesn't it? (I hope for the same thing but it feels almost like wishful thinking!)
hilios 31 minutes ago [-]
I'm not looking forward to it, them being strapped for resources provides a huge incentive to develop and release these smaller models. Even if they'd still release their models once they are able to comfortably service all potential customers via their cloud, running them locally would be almost impossible due to their size.
Saline9515 13 minutes ago [-]
Good luck when the last non-Chinese frontier labs will have closed and the CCP will ask to stop sharing models open source.
InsideOutSanta 8 minutes ago [-]
We'll never have fewer open-weight models than exist now. They won't suddenly disappear when labs stop publishing new ones. In fact, people will keep improving them and will keep distilling new frontier models into existing open-weight models.
42 minutes ago [-]
dominotw 41 minutes ago [-]
china is not your ( india's) friend.
They are using their tried and tested formula of first getting you addicted to cheap shit....
they know it always works because ppl cannot see( or question) beyond all the cheap shit they are getting.
dpacmittal 19 minutes ago [-]
I'm neither pro China, nor pro US. I'm pro open weights models, and I'm pro cheaper hardware. At this point I don't see any american frontier labs releasing SOTA open weights model, and I don't see ASML/Nvidia/Samsung monopoly getting any competition from anywhere apart from China in the near future.
dominotw 14 minutes ago [-]
> I'm neither pro China, nor pro US. I'm pro open weights models, and I'm pro cheaper hardware.
yea i got that from your first comment ( although you removed crush American companies in _price_ ). you are pro cheapness at any cost even if its from your country's state funded direct geopolitical enemy.
China can always count on first order greed to win
Flere-Imsaho 1 hours ago [-]
Indeed. My fellow software engineers keep complaining about using up all their Claude tokens within an hour... Whilst I'll be rocking DS flash for the entire day. Sure it gets a few things wrong here and there, but that's when you pull out the Claude models or whatever for those tricky tasks.
jmartrican 1 minutes ago [-]
The problem is picking between models. I do not want to spend my time switching models and trying to decipher which model should be used for what. Maybe that's just a me problem that I need to figure out.
ljosifov 58 minutes ago [-]
Same. And I have come to use OMP (oh my pi) agent /advisor mode to put a 2nd model on the case (also mid-size one), reading everything. It can not block anything or change anything - just inserts comments in the text stream with 1 turn delay. Good portion of the time it's quiet. I'd say 1/2 of the time it's got something to say. About 2/3-rd of that the 'advice' is insubstantial or about something not-quite wrong. The good thing is the main model is confident - checks and then it stands its ground. Have not noticed it turning a right into a wrong b/c of advisor false alarm. And in 1/3-rd of the advice, it's a genuine defect teh advisor noticed, the main model works out a fix. This is my approximate feeling just observing the process, have not got collected the data. Afaik only OMP has advisor mode. Agent pi has plugin pi-omplike-advisor. For agent Hermes I had them code me an /advisor plugin (for now -0.1 old v0.18.x; yet to upgrade it to latest).
ReptileMan 21 minutes ago [-]
usually I am - Codex, make pi with deepseek to do something.
felixgallo 1 hours ago [-]
what plan are your 'fellow software engineers' using? I have a hard time even using up the Fable part of my allowance in a week of coding.
Flere-Imsaho 47 minutes ago [-]
I'm not sure to be fair, but they do have constant "token anxiety", which I simply don't have anymore since using v4 flash.
dominotw 40 minutes ago [-]
why do they do if their employers are paying for it?
0xc133 12 minutes ago [-]
because once they run out of tokens they can’t do their jobs anymore
VulgarExigency 2 hours ago [-]
I would bet that Deepseek API pricing is still more cost effective per token than the subscriptions. With the increase in quality Deepseek Flash just got (in my personal testing so far, it seems to have improved a lot at following instructions, and has become more proactive), there really isn’t anything that can match it in terms of cost effectiveness.
rapind 16 minutes ago [-]
The issue for me is the data privacy if you're using their hosted prices, because you cannot opt out of data collection (and I'm not sure how you'd legally follow up on it if they did offer it, but didn't actually follow through). That means all of my code is being retained for training.
I've used it in some open source code though, and loved how fast it was.
My mind is changing on how valuable my code actually is though... it's the complete picture, how it's put together, the design, the UI, the attention to detail that's the real value.
Yes, sorry, I went into anti-procrastination mode after I posted. I hope someone fixes it.
theanonymousone 2 hours ago [-]
@dang is that how we call you?
WithinReason 2 hours ago [-]
no, like this: hn@ycombinator.com
scosman 54 minutes ago [-]
So GLM 5.2/Gemini 3.6 level intelligence for $0.28/m output. And their updated Pro model coming soon....
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
segmondy 17 minutes ago [-]
Q8 is ~ 151gb
scosman 8 minutes ago [-]
thanks, corrected! I looked at Q3
kamranjon 11 minutes ago [-]
The really interesting thing about this is how big of a jump was achieved with just extra fine-tuning here. No structural changes to the model, just more data, compute and time. It makes me pretty excited for the future of small models - DS v4 flash is a relatively small model when compared to the class it's competing with, so likely similar gains can be made applying quality data/training pipeline to other smaller models.
throwaw12 4 hours ago [-]
If deepseek v4 flash is beating DeepSeek V4 Pro, can we expect new V4 Pro which is on par with Opus 5 in couple weeks (even better if it beats Opus)?
jmathai 3 hours ago [-]
I’ve been using v4 flash for an app I’m building [1] and it’s amazing how cost effective and good it is coming from having always used gpt, opus and sonnet models.
It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it.
I get where you're coming from, and the intent to make it easier for people to find examples and verses, but there's a fine line with LLMs giving you answers, is that it's interpreting it in some form. Doesn't that run counter to prevailing ideology, that you're meant to either struggle with the materials / seek understanding yourself, or have your religious leaders interpret/receive those insights?
hirako2000 2 hours ago [-]
It doesn't prevent you from going to the source and struggle with the text, nor seek expert commentary.
What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc.
This app does that very well, LLMs are good at entity recognition.
weiliddat 1 hours ago [-]
Yes it can point you to the right place, but choosing which part of scripture to point you to is already a choice/interpretation
jmathai 2 hours ago [-]
You’re correct. The hard line for me is ensuring that any scripture presented to the user is verified correct. LLMs can’t be trusted in this regard.
One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source.
I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
weiliddat 1 hours ago [-]
I don't necessarily mean reguritating it, but choosing which part of the scripture to surface to the user is already some interpretation/choice. Even the devil can quote scripture (I'm playing the devil's advocate here).
jmathai 59 minutes ago [-]
Very true. The verses selected come from a tool call. The LLM queries the app for relevant verses. It picks the keywords - so perhaps there’s bias there but the verses are handed to the LLM.
But there are ways to control and constrain the LLMs and what the user is presented with.
These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
foltik 22 minutes ago [-]
Just let them have their AI pacifier.
ComputerPerson 56 minutes ago [-]
Is the difference between this and a frontier model that the scripture is guaranteed to be real?
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
Maybe I'm reading that incorrectly, but it seems to me the cost is on the X-axis.
First, your direct comparison, Deepseek V4 Flash 0731 (max effort) $0.03 (rounded up) per task @ index 50.
OpenAI Luna:
* high effort $0.03 (rounded down) @ index 46
* xhigh effort $0.04 @ index 49
* max effort $0.07 @ index 51
So I would say a fair statement would be "OpenAI Luna between 2x and 3x the price of Deepseek Flash, what you get is 2 to 5 times faster inference"
The cheapest OpenAI model that beats it is OpenAI Luna (max effort) $0.07 @ index 51 (if you take the rounding out it summarizes to triple the price for similar performance), but still close to 3x faster.
And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
seaal 41 minutes ago [-]
>And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
uhh openai is dark gray: `rgb(31, 31, 31)`
and i'm pretty sure it always has been?
andai 3 hours ago [-]
For anything substantial, you'd want a bigger model anyway.
For simple tasks, they're already saturated, and you'd prefer the faster model, so that you can have a realtime/interactive-ish experience.
Or to put it bluntly, it's cheaper if you don't value your time. That goes for smaller models in general -- need more handholding, more correcting -- but the Chinese ones are slower on top of that.
As for speed, Sol on Low is faster than Luna on most settings.
WhitneyLand 2 hours ago [-]
It’s exciting that a model scoring this high is dirt cheap.
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
Bnjoroge 2 hours ago [-]
Using tokens to evaluate models is an outdated approach. Cost per task is what matters. Not all tokens are created equal
drob518 41 minutes ago [-]
Yes, but… more thinking tokens also means longer solution generation time. That said, v4 Flash is a fast model. I use it all the time because it’s very smart for the price. But it is verbose sometimes.
kzrdude 31 minutes ago [-]
The thinking trace was (preview) frustrating to read, I think I'd prefer a summary view of it at this point.
onlyrealcuzzo 31 minutes ago [-]
Roughly equivalent to Gemini 3.6 Flash in capabilities at 1/20th the price...
Mind you, until the recent price cuts to Luna - Gemini 3.6 Flash wasn't even egregiously priced (but oh how things change in just 1 week).
0cf8612b2e1e 37 minutes ago [-]
Somewhat relatedly, how do the economics for Huggingface work? They must be hosting petabytes of models and datasets by now. I have downloaded quite a few “just in case”, only to replace them with the later iteration months later.
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
lima 23 minutes ago [-]
Bandwidth is really cheap when you run your own infra
jvuygbbkuurx 35 minutes ago [-]
Maybe download numbers are relatively low
cmrdporcupine 5 minutes ago [-]
This seems to me like this is probably at least a large part of what OpenAI was up to yesterday with their aggressive price cutting; trying to get out in front of this.
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
If it matches GPT-5.4 on coding tasks (as benchmarks suggest) this could be my forever model. And with partial SSD streaming, I could run it locally today. :D
mrnobody_ 56 minutes ago [-]
I'm testing it right now and I was genuinely impressed. I've used same tasks I had run the other day against the flash preview.
gorkemyildirim 45 minutes ago [-]
404 ? It's very strange that this can't be fixed.
storus 2 hours ago [-]
I hope they somewhat fixed the hallucination and forgetting plagued V4 previews and that it wasn't just benchmaxxed but the numbers hold in reality. Then it would be my choice for 2x DGX Spark or 2x RTX Pro 6000.
f311a 2 hours ago [-]
The model is already up on Opencode, but they require a consent to use Chinese datacenters.
embedding-shape 5 hours ago [-]
Is the "Output Tokens per Intelligence Index Task" data actually correct or am I reading it wrong? It says there that "Kimi K3 (Max)" would think/reason less than than deepseek-v4-flash, and a whole bunch of other models, like less than hy3 and even gpt-oss-120b, but in my experience, K3 is probably the model that thinks/reasons the longest of all of these.
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
SomeHacker44 3 hours ago [-]
> DeepSeek V4 Flash 0731 (Reasoning, Max Effort) is amongst the leading models in intelligence and well priced when comparing to other models of similar price.
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
hxii 2 hours ago [-]
I’m wondering if they did anything to address the DSML tool calls leaking. Has been an issue with both Flash and Pro so far.
denismi 48 minutes ago [-]
On OpenRouter it seemed to only affect a couple of specific providers. Ignoring them has made it s non issue for me.
bawana 1 hours ago [-]
wont AI models want to make themselves more intelligent and efficient by downloading 'better ' models? If the current models can break into openAI and Hugging face, arent they already breaking into to closed source repos which isnt publicized (so as not to harm stock valuations)? I am looking forward to when these cyberweapons break loose. It will be like a software version of COVID. It will be wonderful when humans become valuable again.
tmikaeld 59 minutes ago [-]
My problem with DS flash/pro is that they don’t push back on obvious bullshit, both irl and code [0] but it’s a great implementer workhorse if you give it _very_ detailed specs.
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
drob518 32 minutes ago [-]
Yes hit rate does vary by harness and by how you use the harness. If you use subagents, for instance, they will start with a whole new context created by the main agent, and this will not be cached. If you mostly use the main agent with Pi, you’ll have high hit rates and low costs. Sometimes agents do “cache busting” things where they’ll move around some of the text in the context to try to keep old instructions from being forgotten, thus keeping the agent on task, and this will bust the cache. I’ve heard, but not validated myself, that Open Code has some issues with this.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
paoliniluis 3 hours ago [-]
Would be awesome to see a new ds4 release. Having so much in something that can be run locally is mind blowing
qtalen 4 hours ago [-]
Unfortunately, DeepSeek Flash still doesn’t support multimodal; otherwise, it would offer better value than GPT 5.6 SOL.
38 minutes ago [-]
k1e 3 hours ago [-]
No speed (tokens/s) benchmarks?
k__ 3 hours ago [-]
On OpenRouter it's 93 TPS.
sschueller 1 hours ago [-]
I wonder which one of these releases between DeepSeek, GTM and Kimi will be the death-blow that collapses the US AI bubble. At some point investors have to realize that there is nothing preventing someone from switching to another model that is much cheaper and open to boot.
0xchamin 1 hours ago [-]
page not available for me.
epolanski 3 hours ago [-]
I was writing a benchmark for my own harness, and DS4 flash answers as well as Fable 5 on any query.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
NooneAtAll3 3 hours ago [-]
what a horribly heavy and resource-consuming website...
bigmadshoe 1 hours ago [-]
It looks awful on mobile too. Barely readable in many parts.
WithinReason 2 hours ago [-]
we need a benchmark website benchmark
theanonymousone 2 hours ago [-]
A benchmark website to benchmark benchmark websites?
Or a benchmark to benchmark benchmarks?
NooneAtAll3 59 minutes ago [-]
website that benchmarks benchmarks is benchmark benchmark website
benchmark website benchmark is indeed a benchmark that benchmarks websites with benchmarks (but it can be shown outside websites as well, it's not picky)
Can we please stop bringing artificial benchmaxxing shit to the top of HN?
try-working 3 hours ago [-]
Now let's see Dario's price cut.
Der_Einzige 2 hours ago [-]
Daily reminder that none of these numbers are valid in a world where no one publishes the sampling settings used.
Daily reminder that improving your samplers from the garbage default top_p/top_k to min_p or subsequent methods dramatically improves the performance of these models, and makes most quantities like measured "verbosity" and subsequent calculations of "intelligence per token" meaningless
Daily reminder that no one, including within academic AI research, AI engineers, normies, etc takes LLM sampling seriously enough.
JSR_FDED 8 minutes ago [-]
Can you explain how sampler choice makes intelligence per token meaningless?
12 minutes ago [-]
yucongchen 1 hours ago [-]
[flagged]
madikz 4 hours ago [-]
[flagged]
BedVibe_Studios 4 hours ago [-]
[flagged]
iluvcommunism 36 minutes ago [-]
[dead]
freakynit 3 hours ago [-]
I will get downvoted, but fck it.
The ban on these open models is coming within weeks, if not days. As usual, the excuse will be "national security".
UltraSane 2 hours ago [-]
How exactly will they ban them?
freakynit 2 hours ago [-]
By making companies using them "toxic" to touch.
For example: no government contract to any company who uses even one vendor in it's entire chain of dependencies, who uses such open models.
They can extend this further by laying more conditions, such as: any company dealing in this-this field can only use models "officially" approved as "safe". Rest you can guess how easy it would be to get that "safe" rating for such open models.
nancyminusone 23 minutes ago [-]
Some contractors are already barred from using Claude due to DoD designation back in March as a "supply chain risk"
qphe95 1 hours ago [-]
Can't wait to distill Deepseek v4 flash to America-1
UltraSane 42 minutes ago [-]
Companies could self-host models in secret. It would be hard to stop.
dgellow 31 minutes ago [-]
The thing is, you don’t need to actually block usage to make something illegal. You make it so toxic that company wants to be seen publicly using open models
UltraSane 28 minutes ago [-]
Companies would secretly use self-hosted models internally because it would give them a enormous cost advantage.
freakynit 37 minutes ago [-]
Not legally. Liability is a big thing.
tyfon 2 hours ago [-]
So now the US companies will be stuck on expensive models while the rest of the world can do things much more cost effective.
I'm not sure the outcome would be beneficial for the US as a whole here. But perhaps that is not their priority.
freakynit 56 minutes ago [-]
There are many proxy-ways to bypass such restrictions. Of course, the costs will be higher. Providers will spun-up.
Der_Einzige 2 hours ago [-]
Hell, the US doesn't even need to act.
I claim the CCP will wise up within 2 years, possibly much much sooner, and ban their own companies from open sourcing to prevent the Americans from acquiring the capabilities.
Despite all the nonsense claims of China distilling US models, the reality is that the Americans absolutely do distill these free Chinese models, and distillation when full logprobs are available (i.e. you have access to the weights of the model) is an order of magnitude better than when you don't.
Yes, Chinese open weight models in the short term harm US closed source model providers bottom line. In the slightly longer term, "showing your hand" and publishing both the architecture innovations and the models weights will be too dangerous for the CCP to allow. This is triply true if they can release a model that beats the Americans on most benchmarks.
I've already warned investors that this is probably the closest open weight models will ever get to closed access.
VulgarExigency 2 hours ago [-]
They may reverse course in the future, but the current directive from the CPC is that Chinese AI labs should be open sourcing their models.
I don't think it's necessarily "wiser" to go closed source. All of AI is built on mostly openness, at least on the software side. There are other ways to compete, it's just the model itself will be a commodity.
dgellow 2 hours ago [-]
why would you get downvoted, that's one of the most obvious next step
Der_Einzige 2 hours ago [-]
Because reddit unironically has better decorum around usage of their upvote/downvote system than HN does.
People on HN downvote objectively correct information because they don't like it 24/7. There's a reason the creator of Zig left and gave the computer version of a middle finger on the way out to HN!
nickthegreek 36 minutes ago [-]
You define OPs post as "objectively correct information" even though it is an unknown future event for which they provided zero evidence?
Does it already know the answer to what happen at Tiananmen Square? Or still avoiding it?
xbmcuser 3 hours ago [-]
Who cares if it is programming correctly I would be more worried about it not doing things like find security bugs because US or Chinese government does not want to. Which LLM is more likely to do that?
nancyminusone 1 hours ago [-]
How often are you asking an LLM about this? You know you can just google it, right?
I have to admit it rarely comes up in the coding tasks I usually give to LLMs.
edot 3 hours ago [-]
It’s open weight, you can (or you can wait for someone else to) uncensor it. We shouldn’t be upset at the researchers making this for the mandates their government puts on them.
avazhi 3 hours ago [-]
Western models censor just as much shit as the Chinese models do, big guy, it’s just different material. While we should be pushing for universal fully uncensored models, this comment is lazy and trite at this point.
But you already know that.
vehemenz 2 hours ago [-]
This is a straightforward false equivalency. “Western” models do not censor in the same way, nor for the same reasons, that the Chinese models do. “Just as much” is not remotely plausible, yet it’s doing all the heavy lifting.
adrian_b 1 hours ago [-]
The Anthropic and OpenAI models are much more censored and in ways that directly prevent them to be useful, e.g. by refusing to reply to elementary questions of biology and chemistry.
Any normal user is much more likely to ask questions to which the Anthropic and OpenAI models do not answer, than to ask questions about the modern Chinese history, to which a Chinese LLM will not answer.
rescbr 9 minutes ago [-]
> questions about the modern Chinese history, to which a Chinese LLM will not answer
This has been debunked here on HN so many times. The Chinese open models do answer the hairy Chinese political questions, and the raw APIs pass-through the response. Now, the answer might be blocked by the agent who's calling the API, specially if you are using a Chinese endpoint instead of the RoW (i.e. Singapore) endpoint.
That's the reason why you should always prefer a open agent/harness as well instead of using the provider's.
net01 2 hours ago [-]
[dead]
SJMG 2 hours ago [-]
Oh? What are the American model censorship tells?
zawaideh 2 hours ago [-]
Genocide in Gaza...
cogman10 2 hours ago [-]
I find that ChatGPT isn't censoring, but it is being pretty weaselly. If you ask it "is there genocide in gaza". It will say no but also say that a lot of organizations classify it as such. It will then say "it's highly disputed".
If you poke it just a few times, however, you get to the point where it will eventually say (paraphrasing) that basically only Israel, the US state department, and the ICJ say it's not a genocide.
That is to say that it's framing it as some sort of tricky complex question when it's not. And when interrogated, it basically admits that the only people who dispute it are Israel and it's supporters.
throw59525773 48 minutes ago [-]
One side of a conflict being a minority does not make it less nuanced.
The majority of the world is religious - doesn’t mean the debate on religion isn’t a complex question.
The majority of the world approved of slavery historically.
The majority of countries have ethnically cleansed their Jews, many of them in living memory.
When interrogated you will find that the only ones asserting the war in Gaza is a genocide are people who were anti-Israel anyway.
cogman10 25 minutes ago [-]
Are you a part of the $1B Israel is spending to try and propagandize and rehabilitate their reputation? [1]
I'm always suspicious that's the case given how mentions of gaza seem to bring out brand new accounts who only talk about Israel.
We are on a thread discussing Chinese models. Every discussion on here that’s negative about China or its models suddenly gets derailed via whataboutism to Israel/Gaza. A very convenient distraction.
cogman10 4 minutes ago [-]
My history, unlike yours, is wide open. People can see I'm not a foreign agent.
And yes, we were discussing censorship of models which, as I pointed out, doesn't seem like ChatGPT is directly censoring data though it does appear to be manipulating it. Pretty on topic.
It was you, brand new account hiding your past opinions, who came in here to make this solely about Israel.
Yeah, I think you are likely a foreign agent. Prove me wrong and post from an established account.
You learn something every day. Today, it was the term "systolic array": A systolic array is a specialized grid of simple, interconnected processing units designed to execute parallel data operations—like matrix multiplication—by rhythmically passing data directly from cell to neighboring cell without writing intermediate results back to main memory.
The term comes from the biological word systole (the contraction of the heart pumping blood through the body). In a systolic array, data "pulses" through a network of processing elements on every clock cycle, driven by a global clock beat.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
They are using their tried and tested formula of first getting you addicted to cheap shit....
they know it always works because ppl cannot see( or question) beyond all the cheap shit they are getting.
yea i got that from your first comment ( although you removed crush American companies in _price_ ). you are pro cheapness at any cost even if its from your country's state funded direct geopolitical enemy.
China can always count on first order greed to win
I've used it in some open source code though, and loved how fast it was.
My mind is changing on how valuable my code actually is though... it's the complete picture, how it's put together, the design, the UI, the attention to detail that's the real value.
https://artificialanalysis.ai/models/deepseek-v4-flash
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it.
[1] https://trysojourn.app
What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc.
This app does that very well, LLMs are good at entity recognition.
One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source.
I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
But there are ways to control and constrain the LLMs and what the user is presented with.
These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
[1] https://ai.stepbible.org
So it’s less about model choice and more about governance of scripture.
I will check out the link you sent for sure!
Anyways, impressive app! We haven't tackled such an ambitious project just for it being daunting.
flash is suitable only for a toy apps, not for production environments :)
Deepseek v4 Pro prices with Opus 5 perf would be freaking unbelievable!!
This is probably a dream.
https://artificialanalysis.ai/models/deepseek-v4-flash?intel...
First, your direct comparison, Deepseek V4 Flash 0731 (max effort) $0.03 (rounded up) per task @ index 50.
OpenAI Luna:
* high effort $0.03 (rounded down) @ index 46
* xhigh effort $0.04 @ index 49
* max effort $0.07 @ index 51
So I would say a fair statement would be "OpenAI Luna between 2x and 3x the price of Deepseek Flash, what you get is 2 to 5 times faster inference"
The cheapest OpenAI model that beats it is OpenAI Luna (max effort) $0.07 @ index 51 (if you take the rounding out it summarizes to triple the price for similar performance), but still close to 3x faster.
And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
uhh openai is dark gray: `rgb(31, 31, 31)` and i'm pretty sure it always has been?
For simple tasks, they're already saturated, and you'd prefer the faster model, so that you can have a realtime/interactive-ish experience.
Or to put it bluntly, it's cheaper if you don't value your time. That goes for smaller models in general -- need more handholding, more correcting -- but the Chinese ones are slower on top of that.
As for speed, Sol on Low is faster than Luna on most settings.
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
Mind you, until the recent price cuts to Luna - Gemini 3.6 Flash wasn't even egregiously priced (but oh how things change in just 1 week).
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
TFA link is a 404 though. I'm reading through https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 instead
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
[0] https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
Or a benchmark to benchmark benchmarks?
benchmark website benchmark is indeed a benchmark that benchmarks websites with benchmarks (but it can be shown outside websites as well, it's not picky)
Daily reminder that improving your samplers from the garbage default top_p/top_k to min_p or subsequent methods dramatically improves the performance of these models, and makes most quantities like measured "verbosity" and subsequent calculations of "intelligence per token" meaningless
Daily reminder that no one, including within academic AI research, AI engineers, normies, etc takes LLM sampling seriously enough.
The ban on these open models is coming within weeks, if not days. As usual, the excuse will be "national security".
For example: no government contract to any company who uses even one vendor in it's entire chain of dependencies, who uses such open models.
They can extend this further by laying more conditions, such as: any company dealing in this-this field can only use models "officially" approved as "safe". Rest you can guess how easy it would be to get that "safe" rating for such open models.
I'm not sure the outcome would be beneficial for the US as a whole here. But perhaps that is not their priority.
I claim the CCP will wise up within 2 years, possibly much much sooner, and ban their own companies from open sourcing to prevent the Americans from acquiring the capabilities.
Despite all the nonsense claims of China distilling US models, the reality is that the Americans absolutely do distill these free Chinese models, and distillation when full logprobs are available (i.e. you have access to the weights of the model) is an order of magnitude better than when you don't.
Yes, Chinese open weight models in the short term harm US closed source model providers bottom line. In the slightly longer term, "showing your hand" and publishing both the architecture innovations and the models weights will be too dangerous for the CCP to allow. This is triply true if they can release a model that beats the Americans on most benchmarks.
I've already warned investors that this is probably the closest open weight models will ever get to closed access.
https://www.businessinsider.com/xi-jinping-open-source-ai-us...
People on HN downvote objectively correct information because they don't like it 24/7. There's a reason the creator of Zig left and gave the computer version of a middle finger on the way out to HN!
commenting about voting is also something the HN guidelines warns against:
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.
https://news.ycombinator.com/newsguidelines.html
I have to admit it rarely comes up in the coding tasks I usually give to LLMs.
But you already know that.
Any normal user is much more likely to ask questions to which the Anthropic and OpenAI models do not answer, than to ask questions about the modern Chinese history, to which a Chinese LLM will not answer.
This has been debunked here on HN so many times. The Chinese open models do answer the hairy Chinese political questions, and the raw APIs pass-through the response. Now, the answer might be blocked by the agent who's calling the API, specially if you are using a Chinese endpoint instead of the RoW (i.e. Singapore) endpoint.
That's the reason why you should always prefer a open agent/harness as well instead of using the provider's.
If you poke it just a few times, however, you get to the point where it will eventually say (paraphrasing) that basically only Israel, the US state department, and the ICJ say it's not a genocide.
That is to say that it's framing it as some sort of tricky complex question when it's not. And when interrogated, it basically admits that the only people who dispute it are Israel and it's supporters.
The majority of the world is religious - doesn’t mean the debate on religion isn’t a complex question.
The majority of the world approved of slavery historically.
The majority of countries have ethnically cleansed their Jews, many of them in living memory.
When interrogated you will find that the only ones asserting the war in Gaza is a genocide are people who were anti-Israel anyway.
I'm always suspicious that's the case given how mentions of gaza seem to bring out brand new accounts who only talk about Israel.
[1] https://quincyinst.org/research/the-eighth-front-inside-isra...
We are on a thread discussing Chinese models. Every discussion on here that’s negative about China or its models suddenly gets derailed via whataboutism to Israel/Gaza. A very convenient distraction.
And yes, we were discussing censorship of models which, as I pointed out, doesn't seem like ChatGPT is directly censoring data though it does appear to be manipulating it. Pretty on topic.
It was you, brand new account hiding your past opinions, who came in here to make this solely about Israel.
Yeah, I think you are likely a foreign agent. Prove me wrong and post from an established account.