It's interesting to hear that Guido is now excited by types. A few years ago his opinion was still that types can't catch most problems. I think I will listen to that interview.
Personally I think types are over-hyped for most (profane) code.
The problem with complicated and/or specific types (which promise to catch more errors) is, in one word, coupling. They create serious dependencies across the whole project or even across project boundaries.
The other problem is: one functions-guy's URL is the next function-girl's string. And all this up- and downcasting usually creates considerable noise. Often overzealous typing is painting oneself in a corner, creating more pain then relief. It's like creating these totally arbitrary object-oriented inheritance-based taxonomies which might make some sense in one place of the code but totally break down in the next place.
But speaking about primitive types like int and str, or maybe list of int - most function arguments can accept only one of those for the code to make sense. So strategically placed type annotations can help reduce some boilerplate and put some useful barriers for error search there. While the simple-types cases are also the easy ones - the first thing you'll notice while testing is usually that you put a str where an int was expected.
> The problem with complicated and/or specific types (which promise to catch more errors) is, in one word, coupling. They create serious dependencies across the whole project or even across project boundaries.
With dynamic typing, you'll still have this coupling, but now it's unchecked and (in practice) undocumented.
Most parts of the code don't actually care about all aspects of the type (or the value), but only that it's an object that they can hand to the next function. Or they only care about a subset of the value's properties (like it being a string, not necessarily a URL).
For example, in Haskell there is the filter function:
filter :: (a -> Bool) -> [a] -> [a]
filter _ [] = []
filter pred (x:xs)
| pred x = x : filter pred xs
| otherwise = filter pred xs
It doesn't actually care what (the type of) a is. This is one of the showcase examples of the power of Haskell's typesystem. (And admittedly it works nicely in this case).
But you can have that easily with no typing at all.
def filter(pred, xs):
return [x for x in xs if pred(x)]
So, dependencies avoided and it's not a huge source of bugs. While in the typed case, a system is needed to prove that the properties that get input come out again (or possibly a function of these properties). That quickly gets quite involved for more advanced cases.
Just look what the average Haskell code looks like, how many language extensions are typically needed, how crippled the code typically is (libraries dictating exactly what properties the client must be able to statically precompute). Most of the Haskell community seems to agree that the end-goal would be a practical system for dependent types (the unification of types and values, i.e. no types at all ;->).
For example, there are some libraries trying to capture the construction and execution of valid SQL queries in the type system. To understand these libraries without doubt a genius-level IQ is required. But the client code is still basically an unreadable mess. And the libraries are not reusable for cases where the database schema isn't known at compile time. The static type system actually prevents code reuse -- it makes the code really inflexible.
That's actually a great argument for static typing! Let's say your distant input component is changed and in some cases doesn't send lists of lists, but one flattened list. With Haskell, the compiler will immediately notify you of this bug, while with Python, you have to hope that your integration tests are sufficiently exhaustive to catch this, or you'll catch it in production. Whoops.
Like I said, the coupling is still there, but it's hidden by dynamic typing.
The problem you're describing has little to do with static or dynamic typing, and more to do with poor development practices.
Regardless of language used, if a person is changing the interface to something (either in internal code, or by upgrading a 3rd party library) then it's their responsibility to make sure the change is propagated to all the places in the codebase that use the interface. It doesn't matter much if they do that by running the compiler, grepping the code, or running a test suite.
What ends up happening in practice instead is that large projects using dynamic languages are hesitant to change anything like that. So if you didn't know the problem domain sufficiently to guess all the interfaces up front correctly, you end up in a pretty bad place.
In contrast, making such changes when you have types is an order of magnitude easier.
The equivalent would be to have integration tests that exercise the entire system with all its functions and code paths checking for basic data compatibility. Seems like a worthwhile thing to have to me, as its not easy to achieve at all.
Not to mention you have a tool to answer "if I change X, what will be affected" - even if you are not changing the type. Just pretend that you changed the type of X (e.g. change "name" to "firstName"), and see all the places that don't compile now. So much better than grep!
>What ends up happening in practice instead is that large projects using dynamic languages are hesitant to change anything like that.
I have seen this "code fear" in both statically and dynamically typed languages and fixed it. When I have worked through it, it has been roughly an equal amount of work to solve it in dynamically and statically typed languages.
Integration tests are usually the key to getting over the code fear hump.
>The equivalent would be to have integration tests that exercise the entire system with all its functions and code paths checking for basic data compatibility.
Integration tests that exercise every relevant user story are a prerequisite for avoiding code fear in any language - statically OR dynamically typed.
In addition to this sprinkling additional type checks around the code helps give confidence that the code is indeed working and gives you additional freedom to refactor. This is something I do when:
A) I encounter a type related error that an assertion would have caught.
B) I was previously confused about what type a var is or if just by looking at it I think somebody else might be confused.
People who have terrible test suites for their code written in statically typed languages are usually insistent that it's best practice to have types checked by the compiler. This means two things about their development practice A) they don't drive development with tests (TDD) and B) they often push unexercised code.
This is a claim I hear often. I don't think its true. Its possible to have integration tests for every user story, yet still not cover even a fraction of the possible code paths due to an explosion of different possible outcomes and inputs.
The great thing about types is that they're guaranteed to cover all code paths. They can't guarantee certain categories of things, but for the things they can guarantee, they easily provide 100% coverage, something that is impossible with integration tests alone.
They are also a tool to greatly reduce the number of allowed inputs and outputs for most code, which in turn greatly reduces the number of tests that actually need to be written. Runtime type assertions only provide one level of that: if you are sending nonsensical input to a method in an uncommon code path not exercised by your integration tests, the runtime type check will probably fail too late (in production). While runtime assertion can tell you if you call your code wrong at runtime, they can't answer "What are all the places that send incorrect input to this method?" Types can do that.
Not to mention that runtime assertions can be costly, unnecessary and very repetitive if added for every method. Comparatively types are free in terms of both performance and programmer effort, especially in languges with good inference.
In short, tests cannot replace types, just like types cannot replace tests. In the areas where they do overlap, types are a way better, more efficient, more tooling friendly choice.
>Its possible to have integration tests for every user story, yet still not cover even a fraction of the possible code paths due to an explosion of different possible outcomes and inputs.
Which is true in any language.
Having a hierarchy of integration tests helps to offset the combinatorial explosion, as does sanity checks that eliminate invalid code paths. These perform a very similar function to static types but you don't have to use them in prototype code and you can add them retrospectively in production code.
>The great thing about types is that they're guaranteed to cover all code paths.
Types are something every programming language has.
>They can't guarantee certain categories of things, but for the things they can guarantee, they easily provide 100% coverage, something that is impossible with integration tests alone.
Inserting type and sanity checks can achieve virtually the same guarantees as static typing when you have a comprehensive test suite.
When you don't have a comprehensive test suite, static typing may seem more attractive but it's a false sense of security.
>if you are sending nonsensical input to a method in an uncommon code path not exercised by your integration tests, the runtime type check will probably fail too late (in production).
Given that you have untested code, you have a higher likelihood of bugs. Period. In statically typed languages too. In dynamic languages those bugs are likely to manifest a bit differently. Yay?
>While runtime assertion can tell you if you call your code wrong at runtime, they can't answer "What are all the places that send incorrect input to this method?" Types can do that.
Arguments that argue best practice that start with with "given that I have poor test coverage..." are not especially convincing.
>Not to mention that runtime assertions can be costly
Runtime assertions are costly in CPU time. Static typing is costly in programmer time. CPUs are cheap, programmers are not.
>unnecessary and very repetitive if added for every method.
Oh yes, which is why I don't add them to every method - just methods where I am convinced they will be useful.
>Comparatively types are free in terms of both performance and programmer effort
Static typing never comes for free and it is often highly inappropriate (which is why many statically typed languages have dynamic typing bolted - e.g. see reflection in Java).
>In short, tests cannot replace types, just like types cannot replace tests.
You were arguing above that static typing was great because it caught bugs in untested code. That certainly sounds like you think it is replacing tests.
> Given that you have untested code, you have a higher likelihood of bugs. Period. In statically typed languages too. In dynamic languages those bugs are likely to manifest a bit differently. Yay?
Yes, test proponents love to say "you are not testing well enough!" Its dishonest, because in practice no system can achieve 100% integration test coverage, so no system is testing well enough :)
> You were arguing above that static typing was great because it caught bugs in untested code. That certainly sounds like you think it is replacing tests.
I never argued that. In fact, I started by saying that what types provide can also be provided by integration tests with that have 100% coverage, but that the second one is a lot more difficult to achieve AND doesn't give the nice extra tooling support (automatic refactoring, extra documentation, instant type incompatibility feedback in IDEs).
Indeed, if you can keep your integration test coverage at 100% in your project, you don't need types. Curious though; are you familiar with an open source project that has that?
Also, 2000 called, they want their static types arguments back. Take a look at modern type systems, especially those that are tailored for dynamic languages (TypeScript, core.typed). They've grown in expressive power and type inference. They've also patched some obviously stupid gaping holes: for example, TypeScript with strictNullChecks will prevent "NPE" bugs.
Most architects of large software projects agree, in my perception. As a counterpoint, I figure that something is wrong if changes routinely affect large parts of the code. The idea is that data is "in the interfaces" instead of global. Except for a few "god objects" changes should be pretty local.
Yes, what is wrong is our understanding of the problem domain. Or even the stakeholder's real understanding of the problem domain :)
Maybe we should stop working on types and focus on easy, high quality languages and tools which will enable other professionals to easily solve problems from their domain doing all the programming themselves. And not just languages - heck, it takes 2 hours just to set up a basic React Native stack for a "Hello World" app.
In most cases you will not be able to write your code in Haskell in the first place. Do try to write a database query construction kit that deals with schemata read at runtime (a very reasonable requirement).
You always have dynamic typing as an escape. If that's what it takes, then that's an option, but I think in practice you'd be able to keep that sort of purity to a small portion of the program.
Haskell is a research language, which sometimes shows in the libraries. I'm not sure if I'd recommend it for general use; I'm more partial to Kotlin these days. It's definitely an interesting approach, though, and anyone who's designing languages ought to make themselves familiar.
>with Python, you have to hope that your integration tests are sufficiently exhaustive to catch this
If there is a border between code I trust and code I don't trust I put some sort of sanity type-checking to catch this type of problem early. This usually gives me immediate notification of certain classes of bug when a test invokes that code.
I also put other forms of sanity checking in that are not type-specific - e.g. check a file exists before passing it on to a method that will read it.
I am not sure if this style of programming entirely mimics haskell's main USP in python, but it sure makes tracking down certain kinds of error much quicker.
Integration tests attack the problem from one direction and strict type checking (whether static or dynamic) attacks the problem from the other direction.
IMHO you have to attack from both directions. Ever increasing integration test coverage is obviously subject to diminishing returns (in terms of bugs caught/investment), but so is ever stricter type systems.
>If there is a border between code I trust and code I don't trust I put some sort of sanity type-checking to catch this type of problem early. This usually gives me immediate notification of certain classes of bug when a test invokes that code.
That's still on a "due dilligence" basis, and too late.
If you are severely lacking in test coverage. Otherwise no, it's not too late.
Slow compilers that take ~2-10 minutes in statically typed languages give slower feedback than test suites that take seconds in dynamically typed languages. This inhibits development speed.
As an abstract statement, I can only agree. In reality, modern compilers can do some incrementally. So when you make a change in file A, the compiler will only change things dependent on A. In practice this chains a small number of file recompiles. The end result is an instantaneous feedback loop.
I get this all the time in Eclipse. Eclipse doesn't really try to do a full compile. It just updates as above. This allows me to see within a few ms if my code compiles. If it doesn't, red line and the reason.
A full compile on a about 40k classes without tests (just compile, static checking) with Maven on my 2013 MBP is about 5 seconds. So still pretty fast. Probably equivalently as a fast as your dynamic language + tests.
The same is true for hot swapping code. The JVM supports this a bit. Spring Boot supports it more. JRebel supports it entirely. When making a micro service or even a website in Spring Boot STS (Eclipse + Spring features), the run time will restart. Clocked time is 1.2 seconds. Not too bad.
For changes to just HTML/CSS/templates/ etc, no reboot time.
> e.g. check a file exists before passing it on to a method that will read it.
Is this not subject to a race condition though? I.e. you check that the file exists, get a green go-ahead flag to execute your method which will read it, by which time some external entity has caused the file to be removed. So then you handle the inevitable errors that are thrown by the OS when you try to access a non-existent file, making the first check redundant.
Or, to provide another example. A recent project I'm working on acts as a control interface for a process running on the same machine, which exposes an HTTP service. When the application starts up, it can check that the process it wants to control is running, but it is inevitable that at some point, the process will get killed while my application is running, and therefore I have to prepare to gracefully handle that case. I may as well unify everything down to that single path of error handling.
>Is this not subject to a race condition though? I.e. you check that the file exists, get a green go-ahead flag to execute your method which will read it, by which time some external entity has caused the file to be removed. So then you handle the inevitable errors that are thrown by the OS when you try to access a non-existent file, making the first check redundant.
It's not intended to catch every edge case. It's intended to act as a sanity check and a barrier to code executing under known invalid conditions.
Point being that if your code is operating under certain assumptions (e.g. config file is present), it makes sense to check those assumptions early and fail fast, hard and clearly if they don't.
>Or, to provide another example. A recent project I'm working on acts as a control interface for a process running on the same machine, which exposes an HTTP service. When the application starts up, it can check that the process it wants to control is running, but it is inevitable that at some point, the process will get killed while my application is running, and therefore I have to prepare to gracefully handle that case.
This is in essence what I was saying: do some sanity checking and report coherent error messages (i.e. was the server even up?) up front before continuing down a code path.
That way you get a clear actionable error early rather than a mess of weird behavior that means you have to play detective to figure out what went wrong.
These super general very reusable functions are just a small part of code. Most of it depends on how arguments objects looks like exactly. It all becomes much harder to maintain the moment the project grows bigger or you are new to it. Types make it much easier to figure out what is allowed as a parameter where and what I am expected to do in order to make the system work. When I switched from typed to untyped language, it was like having hands cut off.
You are making my point, I'm very much in favor of primitive or very simple types. More complex types typically have a bad cost to benefit ratio.
And it's quite doable to get by even without these simple-type annotations. Just put some assertions at strategic locations. Of course there still is some amount of debugging involved. On the other hand development was much cheaper at other places, and once it works you can be reasonably confident that an int will not turn into a string overnight.
The problem with assertions is that sometimes they mean people get woken up in the middle of the night for an issue that would have been caught hours/days/weeks before by a statically-typed language. If you don't have 100% test coverage in Python you just can't rule that possibilility out. I say this as a full-time Python engineer and I've been woken up at 3am by things that slipped through testing and would have been caught in (say) Java or C#.
Now, it is arguable whether it's worthwhile to take that chance to make development quicker and more flexible. I can definitely see that. Also, if you have to wait a long time for the compiler then that might actually be a loss in aggregate.
It does suck trying to refactor large Python codebases though. Running and fixing the tests over and over until errors go down to zero is no fun.
>The problem with assertions is that sometimes they mean people get woken up in the middle of the night for an issue that would have been caught hours/days/weeks before by a statically-typed language. If you don't have 100% test coverage in Python you just can't rule that possibilility out. I say this as a full-time Python engineer and I've been woken up at 3am by things that slipped through testing and would have been caught in (say) Java or C#.
For me, carefully placed assertions combined with a comprehensive (but not 100%), test suite nearly always means A) catching the exception with a test or B) a clear production error that indicates a problem with the production environment.
I've been woken up plenty by poorly written Java that has had its type system intentionally weakened by terrible coders, meaning lots of detective work. It's a fallacy is that dynamic automatically means weak and static automatically means strict.
Assertions aren't just useful in dynamically typed languages. Java would benefit from using them as well.
> You are making my point, I'm very much in favor of primitive or very simple types. More complex types typically have a bad cost to benefit ratio.
What is a "simple type" in your eyes? Primitive types plus aggregates and arrays? What if your data structure is actually complex? Do you just stuff it in a hashmap?
Primitive types plus basic containers mostly. Anything else is just ad-hoc stuff (like algorithms) that should be properly encapsulated, so its representation in a static type system has less utility.
In python, you could type it fairly loosely. Which is more readable, and also helps you catch errors. There's not so much coupling.
from typing import Callable, Iterable
def filter(pred: Callable, xs: Iterable) -> Iterable:
return [x for x in xs if pred(x)]
filter(lambda x: x>1, [1,2,3])
If you tighten it up to only take iterables of ints.
from typing import Callable, Iterable
def filter(pred: Callable, xs: Iterable[int]) -> Iterable:
return [x for x in xs if pred(x)]
filter(lambda x: x>1, [1,2,3])
filter(lambda x: x>1, [[1],[2],[3]]) # this is an error
Float is duck type compatible with int. So if you specify float, it accepts ints and floats. If you specify int, it only accepts ints.
from typing import Callable, Iterable
def filter(pred: Callable, xs: Iterable[float]) -> Iterable:
return [x for x in xs if pred(x)]
filter(lambda x: x>1, [1,2,3])
filter(lambda x: x>1, [1.01, 2.09, 3.34])
The moral of the story is to be as accepting as you can be unless there is an actual need to be very picky about what you accept. (Perhaps if you're doing integer arithmetic, and your code only works with ints, then require ints).
The very beauty and reason behind filter is in that it does not care about its arguments. It is a logical construct - if predicate is satisfied - keep it.
The ADTs, type-tagging, predicate logic and duck-typing are superior and less cluttered than homogeneous box-like variables and restricted homogeneous conditionals are - the cost of static typing.
Yes, one could shot himself in the foot, but one also could run instead of crawl.
Sometimes it is more important to not shoot oneself in the foot than it is to run.
These discussions always end up with two antagonistic camps, so let's acknowledge that we can choose the degree of language support, in keeping a program's semantics as intended, according to the circumstances. Language selection is an aspect of engineering judgement.
It not "appears", it is generic. As generic as it could be. This particular line is also a generator comprehension which produces a lazy sequence.
Run-time errors and so-called null-propagation are well-known issues and once code passed unit tests it is no less trustworthy in principle than statically-typed one. In the first run, yes, there is a chance of a run-time error.
The dynamic languages, notably Lisps, Erlang, Python and Ruby are proved to be superior for quick prototyping and so-called exploratory programming, which has been popularized by pg, along with bottom-up design and the layered-DSLs architecture, in the OnLisp book.
Another classic example is Norvig's Design Patterns in Dynamic Languages which basically ridiculed the whole thing.
There are distinct cultures around MIT Scheme and Common Lisp, Smalltalk and now Python which emphasize expressiveness, minimalism and readability. Such an erudite like you should know this.
>It not "appears", it is generic. As generic as it could be
Only in the sense that accepting arguments that it shouldn't accept and it'll crash with, is part of the general description of the filtering operation -- which is not.
If you prefer, it's "more generic" than it should be. It's not a generic description of the "filter" operation, but a generic description of the "process_input_and_produce_output" operation.
>The dynamic languages, notably Lisps, Erlang, Python and Ruby are proved to be superior for quick prototyping and so-called exploratory programming
Where is that proof published? And what methodology did it follow? Since, you know, we are computer SCIENTISTS and all...
> are proved to be superior for quick prototyping and so-called exploratory programming
Not OP and I don't have proof but I always reach for Python for exploratory programming or a quick "let's try and see what happens". Not sure if types are part of it or not, maybe it is just terseness -- the code is closer to pseudo-code.
I hope one day to learn Rust enough to internalize the type and borrowing system so that I can crank things out just as fast (and hey'd be more reliable and faster out of the door) but I am not there yet.
> Only in the sense that accepting arguments that it shouldn't accept and it'll crash with
Now on that point I think it is not just types. Types are a part of it. C has types and systems written in it crash and segfault all the time. Likewise C++ and so on. Maybe Rust is one new-comer where types and lifetimes would make a difference in practice.
However since OP mentioned Erlang, I'd would it is possible to write more reliable systems in Erlang (Elixir as well perhaps) than in C++ or Java or other such typed systems. I have seen it work in practice, and Ericsson's customers have seen systems work for years with 99.9999999% reliability. Now Erlang has optional types (the more annotations you add the more benefit you get from), but in practice isolated heaps, built-in distribution, solid error logging and reporting, sane concurrency model make a lot more difference.
Back when I was good at Haskell, I would actually use it for exploratory stuff. At the time I think it mostly came down to which standard library I knew better. Python has a huge stdlib, but I actually find prototyping difficult as I don't have my head in the game in terms of what function on X can I use for this purpose, what's the syntax for that again. Python has quite a lot of syntax / language-specific things to know when you think about it. And I've always found the docs hard to decipher.
When prototyping I think I actually want:
1. Little language-specific knowledge you have to learn and remember when you know a lot of other languages. A lot of this is about API consistency. You should be able to recognise a pattern in how the API is organised, and just go from there. Python still doesn't do list.sort(), it's still sorted(list). WHY???? This shit gets in the way every time.
2. Really REALLY good documentation. This is incredibly important. I should go from googling what I want to do or looking up a method to a concise description and an officially maintained usage example in under 10 seconds. Rust is nailing it most of the time here.
3. Types. Trust me to say that, but autocomplete is one hell of a drug, and so is using it to break out of the write-compile-test loop WAY earlier, usually during the write phase.
4. Not necessarily a big standard library, but at least a very good, frictionless package/dependencies system. Rust is getting better at this, the community is even thinking about 'blessed' packages.
Yes, 'proved' was a too strong claim. A few exceptionally good systems has been produced, notably Symbolics CL, Smalltalk, Erlang and Clojure, which after removing all the hype and snowflakery, is a remarkable thing.
No, in principle is exactly when it's a lot less trustworthy than the statically-typed one. Because in principle you could execute some code (untested; data-driven; "is_testing?function(x){return true;}:1.5"; etc.) that passes a non-function instead of a function, and this code would be considered valid (until it goes pop at runtime). But a language that checked this property statically would give an error before execution even started.
> But a language that checked this property statically would give an error before execution even started.
Like C or C++ does.
But you are right - I shouldn't use 'in principle'. It was rather a decorative idiom here, because it is true only for simple functions like map or filter or whatever you take from a standard prelude.
What I mean is that we are not talking about run-time in this context. Yes, one could pass by reference (or even by value) some crap at run-time, but this is a quite different problem. Machine code has this problem too.
This is more a comparison between Python and functional programming than dynamic vs static. This is partly because the language you compared it to is Erlang, which is a dynamically typed language, with no compile-time checking. But also because you can do this superior idiom in statically typed languages if they offer it, and you also get the static type checking.
In statically typed C# with LINQ, you can have, generic over T:
IEnumerable<T> iterable = ...;
var filtered = from item in iterable select item where Predicate(item);
This does not depend on the static typing. It's just syntax.
One non-obvious difference is that the type checker knows that 'filtered' must be an IEnumerable<T>, and you can't accidentally treat it as an IQueryable<T> or an IList<T>, which have subtly different behaviour. You can't, for instance, attempt to sort an IEnumerable without getting it ToList() from it. This is good, because enumerables can be lazy, and lists can't.
Python checks the enumerable/list distinction, but only at runtime. So let's say your Python library offered a method returning a list, and a new version of the library did some filtering before it returned the list.
A test for that functionality might pass independent of the distinction, because what test case attempts to sort a list for no reason? Some other part of your code used to work on the old version, but now breaks, and nobody knows until they run their program or write a really comprehensive test suite.
In comparison, in C#, this problem literally never arises, for one of two reasons:
1. The API initially offered only an IEnumerable<T> and consumers were calling .ToList() before sorting anyway (which is free on an IEnumerable that's actually an IList); or
2. The library author catches the error when it doesn't compile, because you can't implicitly downcast IEnumerable to IList.
Why you are ignoring the fact that one have to type so much less for the same result? ;)
Filter is the canonical example of a truly generic function. Suppose I wish to filter out something from a stream or a port. Then all I shall do is to supply a generic predicate, which only matches what it is supposed to be a predicate of and ignores everything else. This is what a predicate is - a matcher for some category.
So, quickly writing something like
(filter this? (filter (lambda (x) ...) ...))
without even thinking about type signatures is what dynamic languages and type-tagging are all about - quick prototyping.
This is related to the embedding-of-DSLs technique and producing a systems which are layered DSLs instead on a common big ball of mud in Java.
Shall I cite from OnLisp or arc.arc and news.arc? ;)
> Why are you ignoring the fact that you have to type so much less for the same result?
Mostly because in your Python example you're really just _using_ an inbuilt filter function (comprehensions) where in Erlang you're implementing one. If you compare using, it's shorter in Erlang. More importantly, though, it's because I agree with /u/coldtea that one line of statically typed code (like the C# example) does more for you than the equivalent dynamically typed code. It's _not_ the same result.
> a matcher for some category
Well, a type is a category, which is why the study of them is called category theory. Having them helps you write predicates, and helps you avoid trying to match something of an unrecognised category. The type system does the job of 'ignore everything else'. If you're duck-typing and you actually want to handle that job (which most programs don't, they just accept they might get errors), you have to go
Try stringing that together into (filter this? (filter that? list)). My point is, you can totally have that power and expressiveness without forgoing a static type system. LINQ does it, that's enough of an example on its own. Any language worth its salt these days can string together .map(f).filter(pred).reduce((a,b) => a + b) calls and still let you hover over 'b' in your IDE and tell you exactly what it is. You don't have to choose between these things. As for writing all this 'without even thinking about type signatures': I want to think about type signatures when working with complex data. Having the type system actually lets me be more productive, it is not a hindrance. IDEs are a part of that, but also expecting a huge block of code to work first time.
I replied because your comparison purported to demonstrate that only dynamic languages could be so expressive. It was a bad comparison, and also not a useful example of what code that begs for expressiveness looks like. In fact, most static languages allow you to be productive and expressive when you need it, and let you lock down the use of certain code to strict requirements when you want to.
I imagine there is not a thing in the world I could do to stop you from citing OnLisp.
> does more for you than the equivalent dynamically typed code. It's _not_ the same result.
Yes. It does more checking at the cost of imposing restrictions, such as forcing homogeneous containers and conditionals and having less generic, less cluttered code. This is not merely hand-waving, BTW. One just have to take a look at some decent Lisp code, such as arc.arc or some good parts of Common Lisp or Norvig's code from AIMA which is simply wonderful.
As for typing, an old-school ADTs are OK for me (that is constructors, selectors and predicates explicitly defined as procedures). This requires some discipline, because the type system does not do anything for you, but all this is trivial, including writing pattern-matching.
I could argue that strong typing via type-tagging of values (values has a type, not variables principle) is good-enough as long as it comes together with other Lisp's features, such as everything is an expression, everything is a first-class value, which gives one such beauties as the Numerical Tower, but this is quite another topic.
I am also OK with the SML-family languages, love Haskell for its clarity and conciseness and have nothing against them, but... I still think that there are prototyping languages and implementation languages, and I still prefer to prototype in a dynamic-typed language and would still use Common Lisp if I could have a choice or Clojure.
Well, typing less is not necessarily always the most important thing - though it certainly appears to be useful when writing small examples for pedagogical purposes.
You should look at modern structural type systems that support generics. They have none of the flaws you're describing here. TypeScript is one such example.
For example if you define a TypeScript interface
interface Serializable {
toJSON():string
}
then any object that has a toJSON() method that returns a string is automatically accepted as a Serializable. You can even define such anonymous interfaces inline without giving them a name:
function frob(obj:{toJSON():string}) {
// do stuff
}
Basically, you get statically checked duck types and no coupling.
I think Go's story is similar. I know neither TypeScript nor Go, but "no coupling" isn't so simple as removing "implements" qualifiers.
Basically the only way to really eliminate coupling that I see, is passing explicit function dictionaries, and retract the idea that there is only one valid implementation of any given (type, concept) tuple.
It's very bad if all interface implementations have to be physically coupled with the implementation of the class itself (as I think the case is with Java).
It's still at least unflexible sometimes if you are free to put the implementation anywhere (at the class definition site, at the interface definition site, at an independent site) but the system enforces that there is at most one implementation of each interface (this is how it's done in Haskell). This strongly discourages alternative implementations because on has to circumvent the machinery. I don't think there can be a system that automates away at least part of the boilerplate that is dictionary passing, while still retaining the flexibility to choose alternative implementations.
The Haskell (maybe also TypeScript?) way can be very nice if you can be sure that there can be only one valid implementation of a concept. However, most concepts actually don't lend themselves to a single implementation. Most concepts aren't mathematically pure enough for there to be one and only one canonical implementation. Right now I can think only of a few where it's almost always fine to use the default implementation instead of being explicit, because one doesn't care much about the result - like "Show" for debug output.
Take for example JSON: I can think of a thousand ways to create some JSON output from my global data. To get them all in the bag, you would start introducing phantom types and what not, and implement toJSON on those. Really nothing gained at this point, it's only losing some readability at the use site.
Given some run-time introspection or metaprogramming on the other hand, a lot of flexibility can be won back.
Here is an example of my preferred way to do it (metaproramming)
Just spend 5 lines to get the data in shape for the task at hand. The original function dict, having a specific layout convention, and containing functionality that is just not needed here, is specified here (informally)
Its not like its impossible to model metaprogramming with types. TypeScript recently brought a new level of expressiveness and power into the mainstream with mapped types.
Its possible to map over record fields of different types to transform the inner type of a field. Similarly, if I understood it correctly, for your example that would mean you could map an object where every field is of a different but related type Domain<T[Key]> into an object of the same shape containing WsllexFunc<T[Key]> or Decoder<T[Key]>.
Furthermore, you don't have to use Domain<T[Key]> - you can define a more restricted interface that only requires wsllex and decode. Then the extra functionality defined in the Domain<T[Key]> is no longer required, nor mentioned. Similarly the types WsllexFunc<X> and Decoder<X> are simple function types, and any function types with the same shape (arguments and return type) would be accepted regardless of their name. Recursively the arguments and return type are themselves structural, and so on (!)
IMO its incredible how structural type systems change the game completely and we've yet to realise the full implications of this for dynamic languages. My guess is that this is why Guido is excited about types.
Thank you for taking the time (and referring to the point I made instead of criticizing the code, which is experimental and a little crappy in places).
Looks interesting and I will keep an eye on it. But am I right guessing that the TypeScript you linked would not work if the schema is only read at runtime?
Yeah, unfortunately. For something like that, code generation of at least the initial record types would be required for different schemas at before-compile-time. Then they can be transformed further by the type level language.
Another option is to model the type as a dictionary of items, each item being a union of the possible types.
There is a tradeoff here, and I believe that if the rest of the code statically assumes a concrete schema is in place, code generation + record types is the better choice. Otherwise, dictionaries would probably work okay.
There are languages such as Idris and F# that have type providers which can be programmed to read and generate the types from the database, but AFAIK that would still happen at compile-time. https://docs.microsoft.com/en-us/dotnet/articles/fsharp/tuto... - i guess its a kind of "built in" code generation.
Personally I think types are over-hyped for most (profane) code.
The problem with complicated and/or specific types (which promise to catch more errors) is, in one word, coupling. They create serious dependencies across the whole project or even across project boundaries.
The other problem is: one functions-guy's URL is the next function-girl's string. And all this up- and downcasting usually creates considerable noise. Often overzealous typing is painting oneself in a corner, creating more pain then relief. It's like creating these totally arbitrary object-oriented inheritance-based taxonomies which might make some sense in one place of the code but totally break down in the next place.
But speaking about primitive types like int and str, or maybe list of int - most function arguments can accept only one of those for the code to make sense. So strategically placed type annotations can help reduce some boilerplate and put some useful barriers for error search there. While the simple-types cases are also the easy ones - the first thing you'll notice while testing is usually that you put a str where an int was expected.