2025-04-29 Vítor Fróis
Building an intuitive understanding of how Linear Regression works and how it leads to Gradient Descent
Category: Technology
AlphaProof, AlphaGeometry, ChatGPT, and why the future of AI is neurosymbolic
2024-07-28 Gary Marcus
My strong intuition, having studied neural networks for over 30 years (they were part of my dissertation) and LLMs since 2019, is that LLMs are simply never going to work reliably, at least not in the general form that so many people last year seemed to be hoping. Perhaps the deepest problem is that LLMs literally can’t sanity-check their own work.
…
Since LLMs inevitably hallucinate and are constitutionally incapable of checking their own work, there are really only two possibilities: we abandon them, or we use them as components in larger systems that can reason and plan better, much as grownups and older children use times tables as part of a solution for multiplication, but not the whole solution.
…
The idea is to try to take the best of two worlds, combining (akin to Kahneman’s System I and System II), neural networks, which are good at kind of quick intuition from familiar examples (a la Kahneman’s System I) with explicit symbolic systems that use formal logic and other reasoning tools (a la Kahneman’s System II).
https://garymarcus.substack.com/p/alphaproof-alphageometry-chatgpt
How LLMs Work, Explained Without Math
2024-05-05 Miguel Grinberg
https://blog.miguelgrinberg.com/post/how-llms-work-explained-without-math
These bubbles kill cancer
2023-10-06 Jim Lynch
The process [of histotripsy] uses a transducer—which converts electricity into sound—to deliver ultrasound waves to a malignant mass at a precise location. When the waves hit gasses inside cancerous cells, they generate clouds of tiny bubbles through a process known as cavitation.
Pulsing sound waves causes the millimeter-sized bubble clouds to repeatedly grow and collapse. On an ultrasound monitor, it can look like bubbles from boiling water—quickly rising and falling along the surface in your pot.
In the past, researchers saw the creation of bubbles through ultrasound as “uncontrollable,” something to be avoided. Histotripsy, however, generates mechanical energy to activate those bubble clouds and break up the tumor cells’ structure, turning it into a liquid called acellular lysate.
…
histotripsy foils cancer’s cloaking efforts by destroying its cell walls, leaving the tumor antigens in plain sight for the body’s immune system.
https://news.engin.umich.edu/2023/10/these-bubbles-kill-cancer/
A small language model made from matchboxes
2023-09-23 David Corney
… build a text-generating model that is as transparent as possible, with nowhere for sentience to hide.
What Are Dreams For?
2023-08-31 Amanda Gefter
In a series of papers, Blumberg articulated his theory that the brain uses REM sleep to “learn” the body. You wouldn’t think that the body is something a brain needs to learn, but we aren’t born with maps of our bodies
…
In 2013, Blumberg published a paper in Current Biology titled “Twitching in Sensorimotor Development from Sleeping Rats to Robots.” In it, he asked, “Can twitching, as a special form of self-generated movement, contribute to a robot’s knowledge about its body and how it works?”
https://www.newyorker.com/science/elements/what-are-dreams-for
Rules of Thumb for Software Development Estimations
2023-05-01 Vadim Kravcenko
But it is important to capture how long something could take if things go poorly.
Neural Networks, Manifolds, and Topology
Visualizations of low dimension artificial neural networks transforming the input into a representation that can be separated by a line with the different classes of data on each side.
Neural Networks, Manifolds, and Topology
2014-04-06 Christopher Olah
https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
Password advice
There have been news stories about how the Office of Inspector General, U.S. Department of the Interior found weak passwords in the Department of Interior’s Active Directory accounts:
- Passwords Are Terrible (Surprising No One)
- A fifth of passwords used by federal agency cracked in security audit
- A government watchdog spent $15,000 to crack a federal agency’s passwords in minutes/
The OIG report:
P@s$w0rds at the U.S. Department of the Interior: Easily Cracked Passwords, Lack of Multifactor Authentication, and Other Failures Put Critical DOI Systems at Risk
It has password advice on page 8:
NIST SP 800–63 recommends using passphrases instead of passwords …
Password vs. Passphrase Examples
Password = 5pr1ng*Ish3re
Passphrase = DinosaurLetterTrailChance
I believe the passphase words have to be chosen randomly from a large word list to be effective, but it is easier to remember than a complex password.