I think my approach to learning most things I'm interested in is to set a basic goal and learn exactly enough info to get me to that goal. If the goal is too ambitious, I'll set a more reasonable goal and try again. If I'm able to achieve the goal, set a more ambitious goal and learn more during that process. Trying to learn everything about robotics is not a reasonable goal at this time so I'm doing my best to set myself up for success.
The only problem with this approach for learning is that LLMs are meant to be something that you give goals, and it solves the problem. There's no space for learning there. The challenges become stating the goal precisely, and working around the stupidity and idiosyncrasies of the LLM.
Setting tiny goals, getting to them yourself, then setting a slightly larger goal, that's much more intense.
What I mean is that I'm not sure what "success" means in this context. There are already programs that control robot arms. I would think that success would mean that you managed to write a program that would control a robot arm.
You're an experienced programmer although maybe not a physics or mechanical person. Executing this would mean that you learned the mechanics (most people trying this don't have your programming experience, and have to try to do both things at once!)
Learning the mechanics would mean that you would have a good instinct to critique the AI when success in future projects would mean manifesting something you'd never seen before (and being happy you had AI to help you punch above your weight.) This would also give you a good foothold to understand more complex movements and coordination.
At least that's my perspective. The deliverable isn't on the table, it's in your brain. Asking the LLM to do it is like asking the LLM to copy a famous painting. You copy a famous painting so you can learn the movements of the person who painted it, not for the painting itself.
I think this perspective makes sense but a few points miss the mark in my opinion.. In this case, I learned a LOT about how to make an LLM operate outside of a purely virtual environment. In the meantime I also learned some robotics fundamentals that will assist me on my next part of the journey. The idea that LLMs are not a useful learning tool is something that I don't agree on, I can't imagine a better teaching tool than an LLM when used in the correct way.
This is very useful info! I've added the additional 45 degree camera and so far it's seems like there is a regression on skill but I think it's because it wasn't available in the earlier parts of the training.
Can you elaborate on the calibration variables comment? This sounds useful but how would I apply these observations?
This is long and rambly - let's email (see profile) if you want to work through it together.
You're effectively trying to understand how motor inputs change the end hand position, and in particular, you want to know where the table top is so you can position the hand close to it to pick up/ put down.
This means you have some tuning to "learn" before you can apply a control policy / algorithm - and you should be careful how you phrase this so claude/ai can pick up the right vocabulary and bias towards good solutions.
Adding multiple views helps as follows:
0. Measure from multiple views the table top - Keep cameras stead and rigid, and ask claude to use opencv to do multi-view registration so the plane of the table is known precisely. Keep the cameras steady throughout this process - if they wiggle, you can do multi-view registration each measurement...
Paint the "finger tips" bright orange. Not kidding. Use a very flat chess board under the arm for your "workspace". Also not kidding.
1. Move arm to known position, the multiple cameras will measure the april tags movement AND THE FINGER TIP LOCATIONS. The chess board provides a very nice texture. Or a big flat texture of any kind helps here. Since we know the cameras and table positions, we're getting closer to knowing how the arm movements move the hand w.r.t. the table.
2. If your arm has encoders, then manually touch the table at several points, and claude will record the april tags + joint angles + finger locations.
2b. If your arm does not have encoders, then manually touch the table at several points, and record the april tags + finger locations only, but SPECIFY that the arm is now touching the table. --> Claude can use more opencv code to actually locate the touch point on the table. The measurement is noisy, but you will use many movements to figure it out.
Repeat many times, 10-20, multiple touch points. Then, ask claude to do an error analysis and suggest more touch points. Tell it to use system identification techniques / camera/arm calibration techniques. tell it to research these techniques and report errors.
When you're done, you have to do repeatability experiments - the key is this is now automatic. Claude picks joint angles for the arm, the fingers move, the april tags move, the multiple camers calibrate and record positions, and claude repeats. The manual steps are just the bootstrap - you should be automated now.
What you want is claude to output and ORDF of the whole system - bang you can now control the arm using off the shelf software, which claude is happy to set up for you.
When it comes time to pick things up, the multiple views will locate the object precisely and a control network or algorithm can plug in to control it via the ORDF.
What a fantastic comment. The information density alone is impressive, the way in which you took such a complicated process and laid it out in a few paragraphs makes it more so.
There is no attempt to hide it, corporations are rapidly attempting to replace technology workers of all skill levels as soon as possible. We need to protect ourselves as well as we can to secure our futures. Join up and let's act together to better our work lives.
If the days of software engineering are over, then there is very little we can do and other jobs will quickly follow suit. I contend that this will not be the case but corporations are operating under the assumption that workers will be replaced quickly and cheaply. We largely missed the opportunity to organize during the covid pandemic when the balance of power swung towards workers and now it's time to remedy that mistake.
this is incredible, I have been thinking about this exact project for quite some time but a little wary of which approach I might take. Thanks for posting this.
Straightforward is ambiguous. To replicate NanoClaw would probably only take about a day of work and testing and refining in Claude Code, but that's a day I didn't have to spend to get NanoClaw.
Why? That's a decent amount of power for running one thing. It's also dual core! You can emulate early consoles on this microcontroller. Someone made a 3D game for the similar (but less performant) RP2040 running at 250 MHz -> https://github.com/bernhardstrobl/Pico3D
Don't give money to amazon that is better spent on an amazingly efficient postal service. Amazon is subsidized by imaginary money until they put all their competition out of business(including USPS).
My honest question is: If you pull shenanigans like this, isn't it actually making Amazon burn through said imaginary money, thus hastening its demise? The cost of delivering a potato has to be on the order of at least a couple dollars.
Assuming it's the US we're talking about, the federal minimum wage is $7.25, which means that if every worker involved is paid at the minimum wage, you incur a cent of labour costs every 4.97 person-seconds. AFAICT, most Amazon workers are paid substantially higher than the federal minimum wage. And that's just labour costs.
While Amazon is efficient, "fractions of a cent" is probably the wrong order of magnitude for even the most efficient order.
You might be 140 miles round trip to the nearest fulfilment centre, but you're almost certainly closer to your nearest neighbours who regularly buy stuff from Amazon, so the van is probably coming pretty close to you any way.
I think they let you (not YOU necessarily, but the proverbial you.) get away with stuff because they know your habits and you probably make more money for them than you realize.
I can almost guarantee that everyone mentioned in that blog post is a habitual Amazon user. They're all renewing Prime each year at full price and making a ton of regular purchases. The family has even turned on the FOMO by making Prime a family social network with social pressure to stay. I see it as a self-own, personally.
Edit: I'm taking part of this to the root of the thread
Can you explain? Amazon is wildly profitable, and while AWS is far higher margin than their retail businesses, everything I can find suggests their retail segment also has a healthy operating margin.
If you put all of the money Amazon as a whole has taken since it was founded in 1994 in a stack on the left, and all of the money Amazon as a whole has spent since then in a stack on the right, the stack on the left is slightly larger, but this has only been true for a couple of years now.
It's the difference in 1990s billionaires and 2020s billionaires. Bill Gates was so rich because he owned a lot of Microsoft shares and received profits from those shares as dividends. Jeff Bezos is so rich because he owns a lot of Amazon shares and people keep being willing to pay more and more for those shares so his notional net worth increases (AMZN has never paid a dividend).
> his notional net worth increases (AMZN has never paid a dividend).
But that’s exactly the loophole: you can borrow for very cheap against this notional equity without incurring a cent in taxes (since divodends are never paid out)
I hate USPS, and will not be doing anything to benefit them until they offer a way to limit my deliveries to once a month, and opt out of anything that has "or current resident"
At the very least they should charge more for bulk mail, not give out discounts.
In Canada, you can place a red dot (or write no unsolicited mail) on your mailbox and they will withhold delivering anything not directly addressed to you.
I was shocked when I moved to SF and found out there was no way to opt out of unaddressed mail (or "current resident").
In Finland AFAICT there's no bulk postal rate. Instead, paper spam is delivered thru mail slots by private services that hit all the buildings in the neighborhood and drop collections of paper spam. So, many people post a note on their door opting out from this stuff. (Ei mainoksia = No ads.) It must be saving absolutely huge amounts of paper.
The USPS is a government-run spam delivery service that there is no way to opt out of. Those of us who do banking and other administrative tasks online would be better off if the government shut it down completely, or better yet subsidized it slightly so it doesn't have to deliver spam to survive.
But as it is, I don't see any good reason to have any more respect for USPS than I do for any other spammer.
No, it’s completely different. Gmail actively tries to prevent spam. If they catch you sending it, you will be banned, and they let individual users block whoever they want. A huge part of their product is automated spam filtering.
On the other hand, spam delivery is the business model of USPS. They actively and intentionally market and sell their services to spammers, and not surprisingly, give normal users no way to opt out.
There are mail forwarding services[0] that let you automatically filter out junk mail. Yes, they cost money, but at least you can accomplish your goal of opting out (or in) from receiving postal mail from certain senders.
[0] E.g. https://www.usglobalmail.com/virtual-mailbox/
If your goal is to receive zero mail at your address, simply have all your email forwarded to this service and configure it to opt-out of all notifications.
Alternately, you can forward your mail to an entirely different address. (This is just a Gedankenexperiment; please don't spam strangers by forwarding your mail to them.)
>given evolution, one could say "feature, not a bug."
The issue with evolution is huge portions of it just happen to exist and not kill the host before they breed. It could be a massive bug that if corrected could cause the host to breed and spread their genes far further, but evolution itself can't reach there.
>The best thing about a good deep conversation is when the other person gets you: you explain a complicated situation you find yourself in, and find some resonance in their replies.
>That, at least, is what happens when chatting with the recent large models.
The first sentence says a good conversation is between two people. The author then pulls the rug out and says "Psych. A good conversation is when I use LLMs."
The author points out humans have decades of memories but is surprised that when they tell someone they are wrong they don't immediately agree and sycophantically mirror the author's point of view.
The author thinks it's weird they don't know when the next eclipse is. They should know this info intuitively.
The author claims humans have a habit of being wrong even in issues of religion but models have no such flaw. If only humans embraced evidence based religious opinions like LLMs.
The author wonders why they bothered writing this article instead of asking ChatGPT to write it.
Did you ask an LLM if this is satire?
I did and Opus said it wasn't satire.
This was clearly a hallucination so I informed it it was incorrect and it changed it's opinion to agree with me so clearly I known what I'm talking about.
I'll spare you the entire output but among other things after I corrected it it said:
The "repeating the same mistakes" section is even better once you see it. The complaint is essentially: "I told someone they were wrong, and they didn't immediately capitulate. Surely pointing out their error should rewire their brain instantly?" The author presents this as a human deficiency rather than recognizing that disagreement isn't a bug.
I really can't tell to be honest, so if it's satire it's very good. If it's not then I don't really know what it's saying, humans are bad at stuff too? Many humans are not well educated and are not great conversationalists? Does this somehow make LLMs better in our perspective somehow?
If it were satire, what do you think it would be satirizing?
> I don't really know what it's saying
It's saying that complaints about deficiencies in LLMs, about a fundamental lack of LLM intelligence, about how LLMs are just statistical machines and not really thinking, about how LLMs are incapable of learning from past experiences, about how LLMs lack any coherent epistemology ignore how very deficient humans are in many same exact ways.
> Does this somehow make LLMs better in our perspective somehow?
Better is a relative measure not an absolute one, so possibly, because views of LLMs are inherently formed in relation to views of the human brains they're modeling.
> I don't think the article said anything about statistics?
I don't think I said or implied that it did. It's merely one of the many positions that people commonly (and defensively) take for why LLMs aren't and/or can't be intelligent like humans, ignoring that humans exhibit exactly the same patterns.
> If it were satire, what do you think it would be satirizing?
Think of the most terminally online drama you've ever witnessed: the hysterics people work themselves into over what (to outside observers) seems utterly inane and forgettable, the multi-page Tumblr or 4chan posts that become the sacred texts of the "discourse", and the outsized importance people ascribe to it, as if some meme, album cover, or Qanon drop is the modern incantation of the shot heard around the world.
The people wrapped up in this stuff tend to self-select into their own communities because if you're not involved with or amenable to caring about it, why should they spend time talking to someone who will just nod, go "huh, that's wild", and proceed to steer the conversation elsewhere? In their eyes, you may even be a weirdo for not caring about this stuff.
So when I read:
> I’ve got a lot of interests and on any given day, I may be excited to discuss various topics, from kernels to music to cultures and religions. I know I can put together a prompt to give any of today’s leading models and am essentially guaranteed a fresh perspective on the topic of interest. But let me pose the same prompt to people and more often then not the reply will be a polite nod accompanied by clear signs of their thinking something else entirely, or maybe just a summary of the prompt itself, or vague general statements about how things should be. In fact, so rare it is to find someone who knows what I mean that it feels like a magic moment. With the proliferation of genuinely good models—well educated, as it were—finding a conversational partner with a good foundation of shared knowledge has become trivial with AI. This does not bode well for my interest in meeting new people.
I'm imagining the more academic equivalent of someone who got wrapped up in Tiktok drama or Q nuttery but couldn't find a community of kindred souls and, frustrated with the perceived intellectual mediocrity surrounding themself, has embraced LLMs for connection instead. And that's just hilarious. If Silicon Valley was still being produced, I'm sure this would have been made into an episode at some point.
The bits about not generalizing and engaging in fallacious reasoning are also quite amusing since, while yes, the average person likely would benefit from taking (and paying attention in) a couple introductory philosophy classes, expecting all humans to behave logically and introspectively is fantastical thinking.
> expecting all humans to behave logically and introspectively is fantastical thinking
Yes, that is exactly the point of OP's post, that humans are on average quite bad at behaving logically and introspectively and exhibit the very same behaviors that we righteously fault AI for doing. And then OP provides a list of faulty human behaviors that are the same faulty behaviors people give as demonstrating that AI lacks true intelligence.
Meanwhile, AI continues to improve and the human species does not.
And the conclusion is that the fact that the rise of AI has made human faulty behaviors more apparent may creepingly tear at the social fabric.
Read the first paragraph again. It sets the framing through which the rest of the post is understood (as first paragraphs tend to do).
I find this exchange to be a funny example of the truth of OP's list, where the part which sticks with you is some finer detail of one of the examples while the thesis statement itself, the very explanation of the overarching point of the post, seems to have fallen outside of the context window.
The point is that if you have benchmarks for intelligence, which humans would also fail, then you have to concede that either humans are not intelligent, or that the benchmarks are too strict, or aren't a measure for intelligence at all.
The thing is, LLMs would fail that test every time, but humans would pass it most of the time (hopefully). Just because humans are fallible doesn't make LLMs intelligent.
We really haven't got a grip on what intelligence actually is, but it seems that humans and LLMs aren't really in the same ballpark, or even the same league.
>haven't got a grip on what intelligence actually is
Because intelligence isn't a thing, it's a bunch of different things that some intelligent things have more or less (or none of).
This is why measures of intelligence always fail because we try to binary it which doesn't work. Intelligence is spikey. Intelligence scales from very small and dumb to very smart. But even the things that are very smart on a lot of things still do very dumb things. We also measure human intelligence as a function of all humans and LLM intelligence on a particular model.
Why does air-gapped environment require rolling your own CI/CD solution? Plenty of examples of air-gapped Jenkins and/or Argo Workflows. Was this just an educational exercise?
Argo Workflows does not live up to what they advertise, it is so much more complex to setup and then build workflows for. Helm + Argo is pain (both use the same template delimiters...)
Jenkins, like many tools with extreme flexibility, sucks as much as you make it suck. You can pretty easily turn Jenkins into a monstrosity that makes everyone afraid to ever try to update it. On the other hand, you can also have a pretty solid setup that is easy to work on. The trouble is that the tool itself doesn't guide you much to the good path, so unless you've seen a pleasant Jenkins instance before you're likely to have a worse time than necessary.
What particular issues do you have with it? My company uses it at scale (dozens of different instances, hundreds of workers, thousands of pipelines) to support thousands of applications and we are reasonably happy with it. DSL is incredibly helpful at scale. IAC is incredibly helpful at scale. It requires a good amount of upkeep, but all things underpinning large amounts of infrastructure require a good amount of upkeep.
We've minimized our usage of the DSL, there is no way for devs to debug it without pushing commits, and it means you have to implement much of your CI logic twice (once for local dev, once for ci system).
IMO, ci should be running the same commands humans would run (or could if it is production settings). Thus our Jenkins pipelines became a bunch of DSL boilerplate wrapped around make commands. The other nice thing about this is that it prepares you for easier migrations to a new ci system
It's one of the few CI tools where you can test your pipeline without committing it. You also have controls such as only pulling the pipeline from trunk, again, something that wasn't always available elsewhere.
However, it can also be a complete footgun if you're not fairly savvy. Pipeline security isn't something every developer groks.
When was the last time you used Jenkins? I don't get the hate. Not only from you, but lots of people on the internet. What makes Jenkins stand out IMO is the community and the core maintainers, they are perhaps moving slow, but they are moving in the right directions. The interface looks really nice now, they've done a lot of ux improvements lately.
I haven't used Jenkins in a few years, so some of this might change, but in working with it I saw that Jenkins has a few fundamental flaws that I don't see them as working to change:
1. There is no central database to coordinate things. Rather it tries to manage serialization of important bits to/from XML for a lot of things, for a lot of concurrent processes. If you ever think you can manage concurrency better than MySQL/Postgres, you should examine your assumptions.
2. In part because of the dance-of-the-XMLs, when a lot of things are running at the same time Jenkins starts to come to a crawl, so you are limited on the number of worker nodes. At my last company that used Jenkins they instituted rules to keep below 100 worker nodes (and usually less than that) per Jenkins. This lead to fleets of Jenkins servers (and even a Jenkins server to build Jenkins servers as a service), and lots of wasted time for worker nodes.
3. "Everything is a plugin" sounds great, but it winds up with lots of plugins that don't necessarily work with each other, often in subtle ways. In the community this wound up with blessed sets of plugins that most people used, and then you gambled with a few others you felt you needed. Part of this problem is the choice of XMLs-as-database, but it goes farther than that.
4. The way the server/client protocol works is to ship serialized Java processes to the client, which then runs it, and reserializes the process to ship back at the end. This is rather than having something like RPC. This winds up being very fragile (e.g.: communications breaks were a constant problem), makes troubleshooting a pain, and prevents you from doing things like restarting the node in the middle of a job (so you usually have Jenkins work on a Launchpad, and have a separate device-under-test).
Some of these could be worked on, but there seemed to be no desire in the community to make the large changes that would be required. In fact there seemed to be pride in all of these decisions, as if they were bold ideas that somehow made things better.
both the old & new interfaces to Jenkins are riddled with bugs, work seems to be maintenance mode, across the plugin ecosystem too
If you are talking about Jenkins-X, that is a different story, it's basically a rewrite to Kubernetes. I haven't talked to anyone actually using it, if you go k8s, you are far more likely to go argo
It seems like a simple CI/CD in an airgapped environment might be simpler to implement than to (1) learn and (2) onboard an off-the-shelf solution when your airgapped requirement limits your ability to leverage the off-the-shelf ecosystem.
Since you're exercising, you can take it to the next level where you don't specify the next step but the inputs to each task, allowing you to infer the DAG and implement caching...
