swift-evolution VS foundationdb

According to their changelog[0], Clock was added to the standard library with Swift 5.7, which shipped in 2022, at the same time as iOS 16. It looks like static linking by default was approved[1] but development stalled[2].

I expect that it's as simple as that: It's supported on iOS 16+ because it's dynamically linked by default, against a system-wide version of the standard library. You can probably try to statically link newer versions on old OS versions, or maybe ship a newer version of the standard library and dynamically link against that, but I have no idea how well those paths are supported.

0. https://github.com/apple/swift/blob/main/CHANGELOG.md

1. https://github.com/apple/swift-evolution/blob/main/proposals...

2. https://github.com/apple/swift-package-manager/pull/3905

[A Vision for Embedded Swift](https://github.com/apple/swift-evolution/blob/main/visions/e...) has the details on this new build mode and is quite interesting.

> Effectively, there will be two bottom layers of Swift, and the lower one, “non-allocating” Embedded Swift, will necessarily be a more restricted compilation mode (e.g. classes will be disallowed as they fundamentally require heap allocations) and likely to be used only in very specialized use cases. “Allocating” Embedded Swift should allow classes and other language facilities that rely on the heap (e.g. indirect enums).

Also, this seems to maybe hint at the Swift runtime eventually being reimplemented in non-allocating Embedded Swift rather than the C++ (?) that it uses now:

> The Swift runtime APIs will be provided as an implementation that’s optimized for small codesize and will be available as a static library in the toolchain for common CPU architectures. Interestingly, it’s possible to write that implementation in “non-allocating” Baremetal Swift.

I may be out of my depth here as I've only casually used Rust, but this seems similar to Swift's proposed lifetime dependencies[1]. They're not in the type system formally so maybe they're closer to poloneius work

[1]: https://github.com/apple/swift-evolution/blob/3055becc53a3c3...

Swift recently adopted a region-based approach for safe concurrency that builds on Milano et al’s ideas: https://github.com/apple/swift-evolution/blob/main/proposals...

No, it's not. Refcounting CAN be a garbage collection algorithm, but in Swift it's deterministic and done at compile time. Not to mention recently added support for non-copyable types that enforces unique ownership: https://github.com/apple/swift-evolution/blob/main/proposals...

You can stackallocate buffers with unsafe Swift but it's not exactly fun to use. https://github.com/apple/swift-evolution/blob/main/proposals/0322-temporary-buffers.md

Actually, Apple does this for iCloud! They use FoundationDB[1] to store billions of databases, one for each user (plus shared or global databases).

See: https://read.engineerscodex.com/p/how-apple-built-icloud-to-...

Discussed on HN at the time: https://news.ycombinator.com/item?id=39028672

[1]: https://github.com/apple/foundationdb https://en.wikipedia.org/wiki/FoundationDB

Is this the language in question? https://github.com/apple/foundationdb/blob/main/flow/README....

> I wondered if perhaps this WASM use case for a cross-language library was already just as possible and ergonomic using language bindings and maybe thats why this use case doesnt seem like a big deal to people.

Yeah that’s the reason. You don’t notice it a lot of the time, but FFIs are everywhere already. The most common foreign function interface is basically the ability to call C code, or have functions made available to C code. C is used because everyone knows it and it’s simple. And most languages either compile to native code (eg rust) - which makes linking to C code easy. Or the runtime is implemented in C or C++ (eg V8, Ruby). In languages like that, the standard library is already basically implemented via a FFI to C/C++ code.

I’ve got an iOS app I’m working on that’s half rust and half swift, with a touch of C in the middle. The bindings work great - the whole thing links together into one binary, even with link time optimizations. But the glue code is gross, and when I want to fiddle with the rust to Swift API I need to change my code in about 4 different places.

Most FFIs are a one to many relationship in that if you write a clean C API, you can probably write bindings in every language. But you don’t actually want to call naked C code from Ruby or Javascript. Good bindings will make you forget everything is done via ffi. Eg numpy. I haven’t looked at the wasm component model proposal - I assume it’s trying to make this process cleaner, which sounds lovely.

I maintain the nodejs bindings for foundationdb. Foundationdb bindings are all done via ffi linking to their C code. And the API is complex, using promises and things. I find it really interesting browsing their official bindings to go, Java, Python and Ruby. Same bindings. Same wrapped api. Same team of authors. Just different languages. And that’s enough to make the wrapper wildly different in every language. From memory the Java ffi wrapper is 4x as much code as it is in Ruby.

https://github.com/apple/foundationdb/tree/main/bindings

Isn’t apple server on C/C++?

They have their own db in that

https://github.com/apple/foundationdb

> Any meaningful technology is open source.

Clearly untrue, however FoundationDB is open source, with a permissive license.

https://github.com/apple/foundationdb

So is much of the operational tooling for it:

https://github.com/FoundationDB/fdb-kubernetes-operator

I don't recall any of those details but the test involved injecting a bogus block device that always returns garbage, and noting that this results in garbage records returned from client queries. And I don't think those kinds of issues have been eradicated, browsing through their github issues there are people trying to recover corrupted clusters. https://github.com/apple/foundationdb/issues/2480

For my current job, I have to install FoundationDB (they have a github repo) as a dependency for the project's code.

To clarify, FoundationDB has some documented multi-region features, but it's not at all clear that anybody who runs it at scale relies on those multi-region features [1]. Even if they do, it's not obvious they run this Fearless DR mode at any appreciable latency.

Geo-replication is an area where Anna really shines, and foundation pays severe costs. All of the concurrency control problems have contention footprints on the order of round-trips. Fully optimistic concurrency control needs to expose backoffs quite a few round-trips to be live. Even pessimistic concurrency control requires some number of round trips, probably at least 1.5 optimally, but in practice in most tuned systems, probably 3. A heck of a lot of use cases make sense at 0 global RTTs and don't at 1+. The ability to tell the database how to manage concurrency, and then providing causal is ultimately the best you can do. That's awesome.

At the end of the day, I have to believe that there's a whole big mess of applications we'll build this, on systems in the portion of the design space Anna is choosing, one day. This is only recently not cutting edge research, but it's definitely still research. We don't know how to model these concepts at a high-level and in a composable way where the masses of software developers can engage with them.

It's interesting to think about how long ago we were graced with BAYOU [2], that thing was ahead of its time. I suspect it's going to take a little while longer before these sorts of techniques make their way into data storage primitives we think of as part of the standard vernacular, but I believe we'll get there eventually.

Databases are fun!

[1] https://github.com/apple/foundationdb/wiki/Multi-Region-Repl...