mechazoidal + piperesearch   1907

The Be Book - System Overview - The Kernel Kit - Ports
a system-wide message repository, used for queues. usually used with Be's specific 'message' formats. compare to plan9 plumber?
beos  piperesearch  pmz  osdev 
2 days ago by mechazoidal
Emacs Everywhere
"From a broader perspective, it would be best described as a programmable, text-oriented user-interface (containing, among others, an editor). "
"As opposed to other user interfaces Emacs is a programmable environment: any structure, interface element and even code can be passed around and combined between the different interfaces to various programs. "
"EXWM has all the benefits of being Emacs-based with the particularity that all X windows are buffers and as such are part of Emacs buffer list, which makes for the capability to fuzzy-select windows with Helm or Ivy! "Lost windows" belong to the past. When opening several windows at once, you can configure how to display them. (This is a recent addition to Helm.) A nice use-case is to first narrow down some windows using Helm patterns, display them all in an "Exposé" fashion, and finally select your desired windows visually."
piperesearch  emacs  facme  2017 
9 days ago by mechazoidal
On Parallel Programming
"Theoretical frameworks such a Little's Law, Amdahl's Law, and others can help in making design decisions that lend themselves to parallelism. Understanding and exploiting the different sources of parallelism across the hardware/software stack are the key for high-performance implementations. But perhaps most importantly, it is essential to select an architecture and a programming model, which maximizes your chances of exploiting parallelism for your specific workload."

Note that this is definitely not on concurrency, but on parallelism (threads, shared-nothing, etc.)
piperesearch  optimization  programming  reference 
9 days ago by mechazoidal
Leveraging the "Display Server" to Improve Debugging | Arcan
"ways of using the display server infrastructure and its position in the user-space stack, to reason about- and bootstrap- debugging. "

"What I try to refer to is the IPC system or amalgamation of IPC systems that stitch user-space together into the thing you interact with."

"The article is structured around three groups of problems and respective solutions:
- Activation and Output
- Chain of Trust
- Process and Identity"
arcan  piperesearch  pmz  graphics  debugging  ipc 
11 days ago by mechazoidal
Writing a low-level Arcan Client | Arcan
"This is a follow up article to the higher level writing a kmscon/console replacement. In this article, we will instead use the low-level C API to write a simple client. "

PR / pmz: note his description of SHMIF: "intended for extending the engine with features like new input drivers; specialised tools that are supposed to extend the window manager you are running; for tight multi-process integration with the engine scene graph and all of its features. [...] it is comparable to xlib/xcb/wayland-client, but with quite a few more features, and a lot more comfortable to actually use."
pmz  piperesearch  arcan  graphics  protocol 
23 days ago by mechazoidal
The Business Case for Formal Methods • Hillel Wayne
'an "intro packet" you can use to argue for the benefits of formal methods (FM) to your boss. It's a short explanation, a list of benefits and case studies, and a demo. Everything's in TLA+, but the arguments apply equally well to Alloy, B, statecharts, etc. Adapt the material to your specific needs.'

companies/projects:
- AWS
- eSpark Learning
- OpenComRTOS
- Cockroach Labs
- Rackspace
- ElasticSearch
tla  formal_methods  verification  piperesearch  pmz  alloy 
4 weeks ago by mechazoidal
The happinesses and stresses of full-time FOSS work | Drew DeVault’s Blog
"I do have some thoughts that I’d like to share about what kind of stresses I’ve dealt with as a FOSS maintainer, and how I’ve managed (or often mismanaged) it."
lobsters discussion: https://lobste.rs/s/43uca7/happinesses_stresses_full_time_foss_work
piperesearch  advice  opensource  2020 
4 weeks ago by mechazoidal
Unix, Plan 9, and Lurking Smalltalk
good lobsters discussion: https://lobste.rs/s/fw5ubn/unix_plan_9_lurking_smalltalk

"meta-levelfacilities appear less susceptible to design-levelchurn than base-level language features, because they are one step closer to a relatively small set of recurring concepts (whose recurrence though Smalltalk, Unix and Plan 9 I have been documenting). These abstractions appear to sit comfortably as the "waist in the hourglass", supporting diverse surface forms for languages above,and running on diverse hardware-supported "big objects" below"

"Far from being replaced by an all-conquering programming language, one interpretation of the role of operating systems is as an infrastructure providing the mechanisms that allow languages to comeand go, while maximising the composability of the software written using them.Languages themselves may then become "views" onto a space of objects that is managed by the operating system, rather than by a per-language VM"
smalltalk  plan9  pmz  piperesearch  unix  osdev  pdf 
5 weeks ago by mechazoidal
Vulkan is both a good and bad example. Vulkan tries to remove the "bloat" of th... | Hacker News
"Vulkan is both a good and bad example. Vulkan tries to remove the "bloat" of the driver by moving it into the engine (or the middleware the engine uses), which, yes, reimplements a pretty sizable chunk of what the driver used to do. But it exposes the API in a way that requires domain-specific knowledge of how modern GPUs work, which requires, frankly, smarter engine developers. They need to stop only thinking in the ways OGL/D3D taught them to think, and need to also think like a driver developer, or possibly even a compiler developer."

> So where do OpenGL developers move to, when 4.6 moves into "this legacy thing we would like to drop"?
"You implement OpenGL as a middleware that speaks Vulkan. Basically, ANGLE it, but for desktop OpenGL instead of GLES."
piperesearch  opengl  vulkan  discussion  hn  2020 
5 weeks ago by mechazoidal
Work Is Work | codahale.com
"Every time I’ve written or spoken about organizational design, I’ve regretted it. There’s something about staking out a position on it which manages to prove me wrong a few years later. But I’ve been having some long thinks about it again, and here’s what I’ve got. Strap the fuck in."
PR: miight be useful for taking organizational design into account for a system? but might not work at all in a creative environment.
sociology  philosophy  business  organization  2020  essay  piperesearch 
5 weeks ago by mechazoidal
Digesting the Elephant… – Ingo's Blog
The followup paper to "Swallowing The Elephant", about rendering the Disney Moana island dataset
rendering  piperesearch  2020 
5 weeks ago by mechazoidal
Abstract Heresies: Remotely like a procedure call
PR: "Speaking of tail-recursive RPCs, when Google was redesigning its RPC system, I tried to get the new designers to implement a simple tail-recursive capability so that client A could call server B and receive a response from server C. That could have made a lot of common patterns at Google faster, since it was common for A to call B, which called C, which called D, and so on, e.g. to try different mechanisms for handling a particular search query. Unfortunately, I wasn't persuasive. Frankly, it's hard for people who haven't spent much time with tail-recursive languages to see the benefit, and these were hard-core C++ hackers."
piperesearch  functional  rpc 
5 weeks ago by mechazoidal
Nodes - our story
"a JavaScript-based 2D canvas for computational thinking. It's powered by the npm ecosystem and lives on the web. "

it certainly looks neat, but as lobsters notes: "without any info about availability, pricing (if there will be an open source/nonprofit free version), etc. it’s hard to feel any excitement."

This page is probably worth archiving just to show use-cases and how they implemented their debug views.
archive_it  programming  piperesearch  nodejs  graphics 
6 weeks ago by mechazoidal
How to animate SVG signature
PR: note the JS for figuring out length of a stroke
svg  animation  css  webdev  piperesearch 
6 weeks ago by mechazoidal
(Differential datalog) blog/2016-06-21.md at master · frankmcsherry/blog · GitHub
"In this post we will look at a recently re-popularized programming language, Datalog, and how it fits in with differential dataflow. In particular, we will see that:
- Differential dataflow can easily express Datalog programs, which are mostly just map, filter, and join operators with one iterate wrapped around the whole thing.
- The absolute performance of Datalog computations can be as good or even faster using differential dataflow on my laptop as compared to current cluster-based research prototypes from the databases community.
- Differential dataflow enables interactive Datalog computations, both in the sense that
- the source relations can changed arbitrarily with results automatically corrected, and
- top-down queries can be performed without evaluating the entire computation."
(downwides: compile times and cognitive overhead)
datalog  piperesearch  dataflow  rust  2016 
6 weeks ago by mechazoidal
GitHub - heathermiller/dist-prog-book
"Source repo for the book that I and my students in my course at Northeastern University, CS7680 Special Topics in Computing Systems: Programming Models for Distributed Computing, are writing on the topic of programming models for distributed systems."
"This is a book about the programming constructs we use to build distributed systems. These range from the small, RPC, futures, actors, to the large; systems built up of these components like MapReduce and Spark. We explore issues and concerns central to distributed systems like consistency, availability, and fault tolerance, from the lens of the programming models and frameworks that the programmer uses to build these systems."
(note that the main dist-prog-book website is broken in parts as of 2020/01, the github repo is easier to read)
piperesearch  book  distributed  database 
6 weeks ago by mechazoidal
A History of Transaction Histories | Ristret
"I've been trying to understand database transactions for a long time, and recently spent some time researching this with Justin Jaffray. Here’s an attempt to summarize what we’ve learned."
(history of SQL transactions, the difference between Snapshot Isolation and Serializable Isolation, and Peter Bailis re-emphasizing that yes, serializability is Very Important)
piperesearch  database  2018  history 
6 weeks ago by mechazoidal
Things we (finally) know about network queues - apenwarr
"How big should your queue be, and what should you do when it fills up? Many times, we implement or even deploy a networking system before we have answered those questions. Luckily, recent research has given us some guidelines. Here's what we know:"
2017  networking  piperesearch 
7 weeks ago by mechazoidal
Pixelpusher: Real-time peer-to-peer collaboration with React
From Ink & Switch: a overview/explanation of their Pixelpusher collaborative pixel art editor.
piperesearch  from:medium  collaboration  javascript  2018 
7 weeks ago by mechazoidal
GitHub - automerge/automerge: A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.
Ink & Switch: "a library of data structures for building collaborative applications in JavaScript."
Used in:
- Pushpin
- Farm
- Capstone
- Pixelpusher
- Trellis

Limitations: no integrity checking, no security, storage bloat, etc.
javascript  library  crdt  piperesearch  collaboration  repo:github 
7 weeks ago by mechazoidal
Paul Hamilton: History-Based Modeling and Direct Editing
"Most all parametric history-based CAD tools on the market now have some level of direct geometry editing capabilities. Without direct editing, users of history-based CAD tools can only edit preexisting information captured in the structure tree during model creation. Information like sketches, 2D constraints on the sketch, 3D feature parameters and feature order."
"Not all history-based CAD tools do direct editing very well. Direct geometry editing requires functionality from the geometry kernel that typical history-based modeling systems have not had to do. History-based CAD systems are basically Boolean engines that are programmed through the history/feature tree, with the parametric feature being one of the primitives in the Boolean. With direct editing, besides Booleans, the system needs to know how to do local operations, how and when to close gaps, add and remove faces, and extend and trim geometry - all based on actual geometry manipulation. If it doesn't do these things well it will certainly show."
(embedded video shows his example of direct editing)
piperesearch  cad  2012  ui  video 
7 weeks ago by mechazoidal
The State of Mobile XMPP in 2016
Daniel Gultsch(developer of conversations.im) illustrates that XMPP is perfectly viable for mobile devices in 2016, thanks to gradual evolution of the protocol via specific XEPs (0198, 0313, 0352, 0280, 0363, 057)
also a backhand at GCM / ios push stacks: "if you have the money and the resources to design your own push protocol there is simply no need to use the one provided by the vendor."
2016  xmpp  mobile  standards  piperesearch  pmz 
7 weeks ago by mechazoidal
GitHub - warner/magic-wormhole.rs: NOT FUNCTIONAL YET. Slowly porting magic-wormhole to Rust. See wiki for game plan.
"I've spoken to several people about porting Magic-Wormhole from Python to Rust, so here's a repository to get things started. It's likely to be a long road, and I'm personally pretty new to the language, so don't expect things to be working for a while yet."
(note that for PR, pmz, the wiki description of the protocols is more useful: https://github.com/warner/magic-wormhole.rs/wiki )
rust  collboration  piperesearch  repo:github  networking 
7 weeks ago by mechazoidal
Protobuffers Are Wrong :: Reasonably Polymorphic
mostly complaining about the type-system, how the 'backwards/forwards compatibility' isn't, and how they tend to contaminate codebases:
"protobuffers correspond to the data you want to send over the wire, which is often related but not identical to the actual data the application would like to work with. [...] By virtue of their rigid code-generation, manifestations of protobuffers in languages are never idiomatic, nor can they be made to be—short of rewriting the code-generator. But even then, you still have the problem of needing to embed a shitty type-system into the targeted language."
also, interesting comment on dhall language: "Dhall serves a different purpose; it's a highly capable configuration language, but it's kinda lousy as a wire format, and it doesn't have any opinion on questions like schema evolution."
serialization  types  programming  piperesearch  rant  2019 
8 weeks ago by mechazoidal
Is there a way to use logic operators in Blender Shader nodes? - Blender Stack Exchange
"I've spent the past week or so working on basic logic gate implementations in Blender Shader editor. I'm sharing these with the community so anyone who would like to use similar nodes can easily find how to make and use them.
They allow for a little more advanced Shader Node behavior without having to resort to OSL or python scripting."
piperesearch  blender  stackoverflow 
9 weeks ago by mechazoidal
Run your own private CA & ACME server using step-ca - smallstep
"ACME support is widespread, but even more stuff can be configured to use certificates, improving security and reducing your secrets management burden. PostgreSQL, MySQL, Cassandra, CockroachDB, Redis, RabbitMQ, Kafka, gRPC – pretty much everything – can be configured to use mutual TLS for encryption and authentication instead of using insecure connections and shared secrets. All you need is an internal CA powered by step-ca and any command line ACME client to issue certificates."

lobsters discussion: https://lobste.rs/s/v7khum/run_your_own_private_ca_acme_server_using

"one thing this is really useful for is using ACME in development: [...] this should work just like Let’s Encrypt’s staging server but without rate limits, with the ability to issue certs for internal names (e.g., localhost), and without requiring an internet connection in/out."
devops  lets_encrypt  ssl  piperesearch  pmz  security 
9 weeks ago by mechazoidal
Quadrilateral demo - Cassowary Javascript
"The below is a new (as of May 2011) all-javascript implementation of the bounded quadrilateral demonstration that was first released as part of the Cassowary constraint solving toolkit. In addition to Javascript, there are also C++, Java, and Smalltalk implementations, and the C++ implementation is wrapped for use with Guile Scheme (see also Scwm) and Python."
solver  javascript  interactive  piperesearch 
10 weeks ago by mechazoidal
.blend file format of Blender format spec for Kaitai Struct
"Internally, .blend format is a hybrid semi-self-descriptive format. On top level, it contains a simple header and a sequence of file blocks, which more or less follow typical TLV pattern. Pre-last block would be a structure with code DNA1, which is a essentially a machine-readable schema of all other structures used in this file."
blender  reference  format  piperesearch 
10 weeks ago by mechazoidal
Experiments in Constraint-based Graphic Design · cat /var/log/life
"Basalt is implemented as a domain-specific language (DSL), and it’s quite different from GUI-based design tools like Illustrator and Keynote. It’s also pretty different from libraries/languages like D3.js, TikZ, and diagrams. At its core, Basalt is based on constraints: the designer specifies figures in terms of relationships, which compile down to constraints that are solved automatically using an SMT solver to produce the final output. This allows the designer to specify drawings in terms of relationships like 'these objects are distributed horizontally, with a 1:2:3 ratio of space between them.'"

(it's a python DSL for now and very much WIP, but the power of this is amazingly illustrated: replicating hand-made inkscape, Tikz diagrams)
Uses the Z3 solver, but there are some other options discussed towards the end.
lobsters: https://lobste.rs/s/ut5gb7/experiments_constraint_based_graphic
piperesearch  2d  graphics  python  dsl  2019 
10 weeks ago by mechazoidal
CI/CD pipelines: Monad, Arrow or Dart? - Thomas Leonard's blog
"In this post I describe three approaches to building a language for writing CI/CD pipelines."
PR: "It’s convenient to write CI/CD pipelines as if they were single-shot scripts that run the steps once, in series, and always succeed, and then with only minor changes have the pipeline run the steps whenever the input changes, in parallel, with logging, error reporting, cancellation and rebuild support. [...] We can get the same benefits of static analysis by using a monad in an unusual way, here referred to as a 'dart'. Instead of functions that take plain values and return wrapped values, our functions both take and return wrapped values. This results in a syntax that looks identical to plain programming, but allows static analysis (at the cost of not being able to manipulate the wrapped values directly)."
(example ocaml library is at: https://github.com/ocurrent/ocurrent )
ocaml  build  piperesearch  programming  functional 
10 weeks ago by mechazoidal
VFX Reference Platform
"a set of tool and library versions to be used as a common target platform for building software for the VFX industry. Its purpose is to minimise incompatibilities between different software packages, ease the support burden for Linux-based pipelines and encourage further adoption of Linux by software vendors. The Reference Platform is updated annually by a group of software vendors in collaboration with the Visual Effects Society Technology Committee."
(major technologies covered: gcc/glibc, python, qt, numpy, openexr, ptex, opensubdiv, openvdb, alembic, FBX, OpenColorIO, ACES, boost, Intel TBB/MKL, and C++ revision)
piperesearch  standards  programming  reference  linux 
10 weeks ago by mechazoidal
forsyth / styx-n-9p — Bitbucket
"In Inferno and Plan 9, the client-side interface is provided through ordinary system calls: mount, open/create, read/write, close. Outside those systems, library support is needed, and this project is intended to distribute and maintain some of those. The first is a Java client, allowing concurrent access to a remote Styx server, including the use of Inferno public-key authentication. (Secure key storage must be provided separately.)"

note: "We sometimes use the different names to distinguish the authentication scheme: public-key (Styx) or shared-secret (9P) but the scheme is actually separate from the protocol."
java  piperesearch  pmz  repo:bitbucket  9p  inferno  plan9 
10 weeks ago by mechazoidal
In-memory databases
landley is basically summing up the in-memory + log-structured DB stuff(this is from a few years ago)
"For persistence, all you need is an on-disk snapshot of the initial state, plus an append-only log of transactions that change that state. When your database needs to be restarted, you re-read the snapshot and replay the log to update the snapshot. Most of your atomicity and persistence guarantees center around making sure the log entries are committed to disk, but it's all fast streaming writes instead of having to seek all over the disk, and thus has no impact on reads."
[...]
pr: "The obvious way to design an in-memory database is thus in two parts: a direct layer providing function access to the data store with transactions and searching and persistence and such, and the other providing an SQL layer on top of that. Whether the database is in shared memory or not is an implementation detail."
database  piperesearch  sql 
11 weeks ago by mechazoidal
9front usage | Lobsters
josuah's comment is great: "It is surprising how well [remoting into a VM running plan9] works in practice since drawterm got resizing support : all the windows inside it will be resized equally well due to how plan9 handles resizes."
plan9  discussion  pmz  lobsters  piperesearch 
11 weeks ago by mechazoidal
moxie: incremental declarative UI in Rust
note that despite the lobsters discussion, this isn't in Electron, but it does use the DOM and webrender.

there's some great discussion/thinking based on raph's earlier reactive UI post! "my main goal is to identify the technology needed for the analogous ecosystem niche for Rust. Well, that, and I quite like the Hooks API. I have described moxie a few times as "what if a React in Rust but built out of context and hooks?"
piperesearch  rust  ui  gui  dom  optimization  reactjs 
11 weeks ago by mechazoidal
Mergeable replicated data types – Part I – the morning paper
Mergeable Replicated Data Types (MRDTs) are in the same spirit as CRDTs but with the very interesting property that they compose. Furthermore, a principled approach for deriving MRDTs is provided, allowing a wide range of data-types to be lifted to their mergeable, replicated counterparts with very little effort on the part of the programmer. The paper sets the discussion in the context of geo-replicated distributed systems, but of course the same mechanisms could be equally useful in the context of local-first applications.

There’s a lot of material in this paper’s 29 pages, and to make sure I have the space to properly understand the ideas (the process of summary writing is a critical part of that for me), I’m going to spread my coverage over two posts.
crdt  algorithms  distributed  from:morningpaper  piperesearch  multipart 
12 weeks ago by mechazoidal
Dragney Taggart on Twitter: "my kingdom for a monitoring system that lets you visualize ~10K points per second of production latencies in a live, pausable, zoomable graph"
Tweet from @aphyr: Like this is not *impossible* to think about. Riemann has pushed 10M events/sec. I don't think browsers could keep up, but you could imagine doing some sort of weird hybrid rendering thing where you render graph chunks on the server and stream them down to the client as images
piperesearch  visualization  tweets  monitoring 
12 weeks ago by mechazoidal
Amplify Creations
@sinbad recommends the Shader Editor
plugin  unity3d  piperesearch  gamedev 
12 weeks ago by mechazoidal
[zeromq-dev] FFmpeg Adds ZeroMQ Support To Let Multiple Clients Connect To A Single Instance
links to a phoronix story: "improves the streaming options by allowing multiple clients to connect to a single FFmpeg instance without a separate server or multi-cast destination address setup as previously required."
2019  zmq  ffmpeg  piperesearch 
november 2019 by mechazoidal
Nvim documentation: api
On nvim's API: discoverability, channels, etc.
neovim  piperesearch  api  lua 
november 2019 by mechazoidal
We Can Have Nice Things: Neovim and the state of text editor art in 2019
I admit, neovim is looking pretty tempting at this point, with lua+easy GUI+vim mentality.
PR: "Nvim audience is "people who want more potential + less entropy". YMMV."
"IOW: ecosystems tend to be winner-takes all (80% of users will use the top few, the rest is "long tail") cf. textmate grammars, javascript, Vim plugins, ..."

"- Ship half of the Right Thing so that it spreads like a virus.
- Then take the time to improve it to 90% of the Right Thing."
(getting an idea for for "mimic the good parts of Blender script/function interfaces")

also, rebuilding their own terminal/GUI since they can't depend on the OS anymore for GUI
"Writing a GUI with Neovim is crazy easy. It took me about 4 hours, including learning a GPU framework."
"Lua's lack of "batteries included" is a benefit. Nvim is the "stdlib"."
facme: vim.treesitter: query the syntax tree
neovim  presentation  slides  pdf  vim  lua  piperesearch  facme 
november 2019 by mechazoidal
The Concurrent 2-Trie - Chris Vest - Medium
"The concurrent 2-trie is a dictionary-like data structure that is designed and optimized specifically to be used for translation tables in file buffer pools. When the concurrent 2-trie replaced the lock-striped hop-scotch hash tables in the Neo4j file buffer pool, file buffer accesses became 30% faster."
"As simple as this data structure is, I have not yet found a description of it in literature. Hence I decided to do this write-up. If this reminds you of a paper you have read, then feel free to send me a link."
database  piperesearch  datastructures 
november 2019 by mechazoidal
Safespaces: An Open Source VR Desktop | Arcan
"the current stages of work on a 3D and (optionally) VR desktop for the Arcan display server. It is tentatively called safespaces as an ironic remark on the 'anything but safe' state of what is waiting inside. For the impatient, here is a video of it being used "
arcan  vr  gui  2018  piperesearch 
november 2019 by mechazoidal
An interview on what makes Postgres unique (extensions) - Craig Kerstiens
PR: not so much for postgres, but this quote: "Extensions have allowed Postgres to extend beyond a traditional relational database to much more of a data platform. [...] Extensions allow the core to move at a slower pace, which makes sense. Each new feature in core means it has to be thoroughly tested and safe. That’s not to say that extensions don’t, but extensions that can exist outside core, then become part of the contrib provide a great on ramp for things to move much faster."
postgres  interview  piperesearch  2019 
november 2019 by mechazoidal
SRFI 167: Ordered Key Value Store
"This library describes an interface for an ordered key-value store that is suitable for implementing a storage engine for the generic tuple-store SRFI. It maps cleanly to existing ordered key-value databases that may or may not provide transactions."
PR: "The author argues that getting started with data management systems with ordered key-value stores is easier to the mind and also reflects the modern industrial practice that builds (distributed) databases systems possibly with SQL support on top of ordered key-value stores. Otherwise said, this SRFI should offer grounds for apprentices to learn about data storage. It will also offer a better story (the best?) for data durability in Scheme implementations. [complements SRFI 168: Generic Tuple Store]"
scheme  reference  piperesearch  database 
november 2019 by mechazoidal
version control - What does Linus Torvalds mean when he says that Git "never ever" tracks a file? - Stack Overflow
PR: "In my taxonomy the term for what we're interested here is the 'atomicity' of the version control system. [...] When a VCS has file-level atomicity, there is in fact a history for each file. The VCS must remember the name of the file and what occurred to it at each point. Git doesn't do that. Git has only a history of commits—the commit is its unit of atomicity, and the history is the set of commits in the repository. "
"Note that a VCS can be commit-oriented, yet still store data file-by-file. That's an implementation detail, though sometimes an important one, and Git does not do that either. Instead, each commit records a tree, with the tree object encoding file names, modes (i.e., is this file executable or not?), and a pointer to the actual file content. [and contents in blobs]"
"The file history you see [with 'git log'] does not exactly exist in the repository, in some sense: instead, it's just a synthetic subset of the real history. You'll get a different "file history" if you use different git log options!"
(there's also his VCS book linked)
piperesearch  git  stackoverflow  answer  dvcs 
november 2019 by mechazoidal
File systems unfit as distributed storage backends: lessons from ten years of Ceph evolution – the morning paper
"Ten years of hard-won lessons packed into just 17 pages (13 if you don’t count the references!) makes this paper extremely good value for your time. It’s also a fabulous example of recognising and challenging implicit assumptions. In this case, the assumption that a distributed storage backend should clearly be layered on top of a local file system. Breaking that assumption allowed Ceph to introduce a new storage backend called 'BlueStore' with much better performance and predictability, and the ability to support the changing storage hardware landscape. "
from:morningpaper  ceph  piperesearch  distributed  filesystems 
november 2019 by mechazoidal
call-with-current-continuation.org: Strand
"an open-source implementation of the identically named parallel Prolog dialect as described in the book "Strand: New Concepts for Parallel Programming" by Ian Foster and Stephen Taylor. Strand represents a very elegant and concise notation for expressing concurrent and distributed algorithms using very few fundamental language constructs."

"This implementation of Strand is written in Forth and runs on Linux x86-64/ARM, Mac OS X x86-64 and OpenBSD amd64 systems, has no dependencies and is entirely self-contained."

PR: it could almost be thought of as Erlang's long-lost junior sibling(prolog syntax, automatic vm communcation(albeit via shared-memory/files). Confirmed on lobsters!: https://lobste.rs/s/04vqwy/implementation_strand_parallel_prolog#c_9fynpo

Might be useful to look at for the prolog concepts and automatic topology(rings, torus), but the software itself is probably too young.
prolog  forth  piperesearch  language  programming 
november 2019 by mechazoidal
Ulf Wiger » What is Erlang-Style Concurrency?
- Fast process creation/destruction
- Ability to support >> 10,000 concurrent processes with largely unchanged characteristics.
- Fast asynchronous message passing.
- Copying message-passing semantics (share-nothing concurrency).
- Process monitoring.
- Selective message reception.

PR: "If there is any single defining characteristic of Erlang-style Concurrency, it is that you should be able to model your application after the natural concurrency patterns present in your problem. If creating a process is perceived as expensive, programmers will reuse existing processes instead; if message passing is perceived to be expensive, various techniques will be invented to avoid sending messages. These workarounds are nearly always detrimental."
piperesearch  erlang  concurrency  2008  asynchronous 
november 2019 by mechazoidal
GitHub - processone/ejabberd: Robust, Ubiquitous and Massively Scalable Messaging Platform (XMPP, MQTT, SIP Server)
stackoverflow: "one of the most well know erlang application and the one I learnt erlang with. I think it's the one of most interesting project for learning erlang because it is really building on erlang's strength. (However some will argue that it's not OTP, but don't worry there's still a trove of great code inside...)"
erlang  xmpp  repo:github  reference  piperesearch 
november 2019 by mechazoidal
Interprocess Locks | Squadrick
"ROS uses a publish-subscribe paradigm for communicating between messages, and internally it uses either TCP/IP or UDP, which are both quite slow compared to using, say, the system’s shared memory. I decided to build a small simple IPC library that uses the same pub sub pattern, but using shared memory instead."

https://github.com/Squadrick/shadesmar/blob/master/include/shadesmar/ipc_lock.h

PR: "memory copies from GPU to shared memory and vice-versa, without the need to do an intermediate copy to the process’ memory space." He used Boost/C++, but it should be able to do this in Rust as well?
piperesearch  ipc  concurrency  ros  c++ 
october 2019 by mechazoidal
Kubernetes and the Erlang VM: orchestration on the large and the small « Plataformatec Blog
"If you are using Erlang/Elixir and you wonder how Kubernetes applies compared to other languages, you can use Kubernetes for the Erlang VM as you would with any other technology. Given that Erlang/Elixir software can typically scale both horizontally and vertically, it gives you many options on how you want to allocate your resources within K8s. On other areas, Kubernetes and the Erlang VM can nicely complement each other, such as using K8s Service Discovery to connect Erlang VM instances. Of course, Distributed Erlang is not a requirement and Erlang/Elixir are great languages even for stateless apps, thanks to its scalability and reliability."
PR: "As users connect and disconnect and as nodes are brought up and down, you could somehow update the database or communicate via a complex RPC mechanism, while carefully watching the cluster for topology changes. With the Erlang VM, you can just broadcast or exchange this information directly, without having to worry about serialization protocols, connection management, etc, as everything is provided by the VM. All without external dependencies. "
erlang  k8s  distributed  piperesearch  devops 
october 2019 by mechazoidal
Generative Artistry
Tutorials and podcasts about generative art, including history. Tutorials are fully interactive(usually using the canvas element) and include Hypnotic Squares, Piet Mondrian, Vera Molnár, and more!
webdev  css  art  javascript  interactive  tutorials  podcast  piperesearch  graphics 
october 2019 by mechazoidal
Pipecut Project Home
"Pipecut tries to facilitate the development of pipelines by letting you see your data and your shell commands at the same time, eliminating the back and forth editing cycle of entering and quitting more(or less), then recalling and editing the command line. Since pipecut has an AST view of the Unix command line, it can provide shortcuts, optimizations, and do code generation that would not be possible otherwise."
last updated 2015, see https://github.com/akavel/up for a modern (accidental) reinvention
piperesearch  unix  pipes  tools  visualization  shell 
october 2019 by mechazoidal
Blobs in Games: Verb-noun vs noun-verb
Starts with interesting stuff in the Ultima games, then goes into vim and auto-complete.
"In games it seems like it'd be better for players to first choose an object from the environment and then choose from a small set of actions, than to first choose from a large set of actions and then choose from a set of objects. However I haven't surveyed enough games to see what's more common. The next time you're playing a game, look at the structure of commands to see if it's verb-noun or noun-verb. "
gamedev  vim  facme  piperesearch 
october 2019 by mechazoidal
Pointer Based Data-Structures in SPARK - The AdaCore Blog
"The specification and verification of pointer-based data-structures is a challenge in deductive verification whether the 'pointer' part is implemented as a machine pointer or as an array index (as an example, see our previous post about verifying insertion inside a red-black tree). In addition, SPARK has a strict ownership policy which will prevent completely the use of some (doubly-linked) data-structures, and complicate the writing of usual algorithms on others. However, I think I have demonstrated in this post that is still feasible to write and verify in SPARK some of these algorithms, with comparatively few user-supplied annotations."
Note how they're borrowing some ideas from Rust.

See also: https://lobste.rs/s/tj7bom/pointer_based_data_structures_spark
ada_language  ada  programming  verification  pmz  piperesearch 
october 2019 by mechazoidal
An Introduction To Real-Time Subsurface Scattering
'If you were to ask me right now to choose between the 3 techniques that I discussed, I would probably go with the screen-space option. In general it seems to have the least corner cases to handle, and most closely resembles a ray-tracing approach. But that could also be a bit of "the grass is always greener" syndrome on my part. I also fully expect a few games to start using ray-traced SSS on high-end hardware at some point in the near future, since you can use that for high-quality results while keeping a screen-space path as a fallback for weaker hardware'
gamedev  rendering  raytracing  piperesearch  textures  shaders  2019 
october 2019 by mechazoidal
The Fascinating Influence of Cyclone
The history of Cyclone, what it influenced, and who was involved. Covers Ada, Midori, M#, Pony, Rust, and others

PR: "For all their benefits, unique (linear) references have a significant limitation. Using only unique references, only hierarchical (tree) data structures can be built. Nearly all data structures with cycles need to allow multiple references to the same object, which unique references cannot do. This limitation, as well as the disruptive constraints of move semantics, should stay top-of-mind when touting the virtues of single-owner memory management and unique references."
programming  safety  history  c  c++  piperesearch  memory 
october 2019 by mechazoidal
GitHub - crawshaw/sqlite: Go SQLite3 driver
Not actually a DB driver, but an attempt to bring some of the SQlite non-db features into the interface via C linking.

See also: https://crawshaw.io/blog/go-and-sqlite
sqlite  golang  repo:github  database  piperesearch 
october 2019 by mechazoidal
Modern C for C++ Peeps
Good stuff for C99:
- compound literals, designated initialization
- don't be afraid to pass-by-value!
- named optional arguments
- easier ways to avoid pointers for data blobs: use tagged index handles, embrace "dumb data, smart functions"!

see also: https://lobste.rs/s/ktl6cs/modern_c_for_c_peeps
archive_it  c  programming  reference  piperesearch  pmz 
september 2019 by mechazoidal
Autodesk Graveyard - cad nauseam
"This page lists some of the Autodesk products that are no longer sold or for which end-of-life or no-more-development have been announced. This is provided as-is for historical interest and with no guarantee of accuracy. "
(last updated 2018?)
autodesk  inspiration  piperesearch  reference  software 
september 2019 by mechazoidal
Latacora - Cryptographic Right Answers
"There are, in the literature and in the most sophisticated modern systems, “better” answers for many of these items. If you’re building for low-footprint embedded systems, you can use STROBE and a sound, modern, authenticated encryption stack entirely out of a single SHA-3 like sponge constructions. You can use NOISE to build a secure transport protocol with its own AKE. Speaking of AKEs, there are, like, 30 different password AKEs you could choose from.

But if you’re a developer and not a cryptography engineer, you shouldn’t do any of that. You should keep things simple and conventional and easy to analyze; “boring”, as the Google TLS people would say."
cryptography  reference  2018  piperesearch  security 
september 2019 by mechazoidal
oil/regex-manual.md at master · oilshell/oil · GitHub
"Egg Expressions (Oil Regexes)"
"The language is designed for "dumb", one-to-one, syntactic translations. That is, translation doesn't rely on understanding the semantics of regexes. This is because regex implementations have many corner cases and incompatibilities, with regard to Unicode, NUL bytes, etc."
"Eggexes are meant to be used with existing regex engines. For example, you translate them to a POSIX ERE, which is executed by egrep or awk. Or you translate them to a Perl-like syntax and use them in Python, JavaScript, Java, or C++ programs."
regex  oil_shell  piperesearch  repo:github 
september 2019 by mechazoidal
So you want to write a package manager - sam boyer - Medium
Most of the article is about package managers general, the Go stuff is only at the end.
PR: note the "project specific manager", and then think about not only user scripts being part of the"project", but also textures and any other external files. Versioning as well: the Armstrong method of using a hash of the source code is an approach, but that's tricky for mere mortals to grok!
Golang  packaging  build  piperesearch 
september 2019 by mechazoidal
The Power of “Yes, if”: Iterating on our RFC Process — Squarespace / Engineering
"We took three steps, each of which incrementally improved our process: we wrote an opinionated RFC template, created Infrastructure Council, and introduced Architecture Review."
(PR: "list why existing solutions didn't work for you")
collaboration  software_development  piperesearch 
september 2019 by mechazoidal
GitHub - spacejam/tla-rust: writing correct lock-free and distributed stateful systems in Rust, assisted by TLA+
However, it needs to be noted that when creating lock-free high-performance algorithms, we are going to need to sidestep the safety guarantees of the compiler. Our goal is to create data structures that are mutated using atomic compare-and-swap (CAS) operations by multiple threads simultaneously, and also supporting reads at the same time. We choose not to sacrifice performance by using Mutexes. This means using Rust's Box::into_raw/from_raw, AtomicPtr, unsafe pointers and mem::forget. We are giving up a significant benefit of Rust for certain very high-performance chunks of this system. In place of Rust's compiler, we use the TLA+ model checker to gain confidence in the correctness of our system!
rust  tla  repo:github  formal_methods  piperesearch 
september 2019 by mechazoidal
Nvim documentation: ui
Showing how vim's internal state is available over a defined protocol
piperesearch  facme  vim  neovim 
september 2019 by mechazoidal
« earlier      
per page:    204080120160

related tags

2d  3d  3ds  3dsmax  3d_printing  9p  9slice  abi  abstract  abstraction  acm  acme  activitypub  actor  actors  actor_model  ada  ada_language  addressing  adobe  adt  advice  advocacy  aesthetics  afl  agile  ai  alf_language  algebra  algol  algorithims  algorithms  aliceml  alloy  ama  amazon  ampl_language  analysis  analytics  anatomy  android  animation  anime  answer  antifragile  antipattern  apache  api  apl  app  apple  ar  arcan  architecture  archive  archived  archive_it  array  array_language  arrows  art  article  articles  arxiv  asm.js  asn1  assembly  assimp  asspad  async  asynchronous  audio  autodesk  automation  avisynth  awk  aws  backup  bash  basis_codec  beaker  beos  bestpractices  bezier  bibliography  big-data  bioinformatics  bitcoin  blender  blockchain  blog  bloom  bluetooth  book  books  brep  bret_victor  browser  build  burnout  business  c  c#  c++  cache  cad  calculator  calculus  calm  camera  canvas  cap  capitalism  capnproto  cartoons  cbor  celshading  cel_shading  ceph  certificate  cg  cgal  chat  chrome  cinematography  citeseer  class  cli  climate_change  clipstudio  clock  clojure  clojurescript  cloud  cmake  code  codec  codepen  cola  collaboration  collboration  color  comics  comments  communication  community  comparison  compilers  completion  complexity  components  compression  computation  computers  computerscience  computer_vision  computing  concept  concurrency  concurrent_ml  confidentiality  config  configuration  consistency  constraint  constraints  containers  contract  controller  converter  coordination  corba  cqrs  crafts  crdt  creativity  crev  criticism  critique  cryptography  csg  csp  css  culture  curves  customization  d3  d3d  dadgum  dance  danluu  darcs  data  database  dataflow  datalog  datascience  datasets  datastructure  datastructures  data_frame  data_source  datomic  dat_project  DAW  dbms  dead_link  debugging  decentralized  deconstruct  demo  demoscene  dependencies  deprecated  design  designer  design_patterns  desktop  development  devops  dht  diff  direct3d  directx  disassembler  discussion  distributed  distribution  djb  documentation  docuverse  dom  dotnet  drawing  drdobbs  dreamcast  dropbox  dsec  dsl  dsp  dvcs  ebook  eclipse  ecs  ed  editor  editors  edn  education  ef-12  electronics  elixir  elm  elm_language  emacs  email  embedded  emulation  emulator  encryption  engine  engineering  erlang  essay  ethernet  etoile  eureka  events  event_sourcing  event_streams  eve_language  example  excel  exokernel  explanation  exploit  f#  facebook  facerig  faces  facme  faq  Farbrausch  fbx  federation  fediverse  fedwiki  ffmpeg  fightman  figure_drawing  file  files  filesystems  filmmaking  flash  flash_memory  flickr  fonts  formal_methods  format  forth  fortran  forum  fossil  fp  fpga  fractal  framework  free  freebsd  freedom  french  frep  from:acm  from:ann  from:aosa  from:cgworld  from:citeseer  from:eff  from:gamasutra  from:hackaday  from:ltu  from:medium  from:morningpaper  from:msresearch  from:nodontdie  from:stackoverflow  from:techcrunch  from:wikipedia  frontend  frp  fsm  functional  futurism  ga  gadt  gallery  gamasutra  game  gamedesign  gamedev  games  gdc  generation  generator  geography  geometric_algebra  geometry  gist  git  github  glitch  glsl  gltf  gnome  golang  google  googledocs  gpg  gpgpu  gpl  gpu  grammar  graph  graphics  graphql  graphviz  graph_theory  gstreamer  gui  guitktk  h264  hack  hadoop  hahaonlyserious  hardware  haskell  HATEOAS  hex  hg  history  hlsl  hlvm  hmd  hn  hosting  howto  html  html5  http  humor  hypercard  hypermedia  hypertext  ibm  icons  identification  identity  ietf  illustration  imgur  immutability  impromptu  indie  inferno  infoq  innovation  input  inspiration  integration  interaction  interactive  interface  internet  interpreter  interview  ios  IoT  ipc  ipfs  iso  itch  izware  japan  japanese  java  javascript  jit  jobs  journalism  json  julia_language  jvm  j_language  k8s  kafka  kakoune  kdb  kernel  khronos  kinect  kotaku  kragen  k_language  lab  labor  landscape  lang:jp  language  languages  lasp  latency  layout  learning  lecture  legal  lego  lens  lets_encrypt  leveldb  libraries  library  licensing  lighttable  lightwave  linda  linear_algebra  linkedin  linux  lisp  list  literature  live2d  livecoding  llvm  lmdb  lobsters  logging  logic  logo  longread  lotus_notes  lua  luajit  machine_learning  machinima  macros  macross  make  makefile  management  manga  manifesto  mapping  maps  marx  mastodon  math  mathematics  matrix  maya  mdns  meander  mecha  media  medium  memcached  meme  memory  mercury_language  mesh  messaging  metadata  methodology  metrics  microservices  microsoft  midi  midori  mindmap  minikanren  minix  misfeature  mixer  ml  mmd  mmo  mobile  model  modelchecker  modeling  models  monitoring  motion  motion_capture  mouse  movies  mozilla  ms_research  mud  multimedia  multipart  music  mvc  nanomsg  nanovg  nasa  neovim  network  networking  networks  newton  nintendo  nix  nlp  node  nodejs  notation  notes  npr  ntp  oats  objective-c  ocaml  occam  ogg  ohm  oil_shell  om  ometa  oo  oop  openbsd  opencl  opencv  openfl  openframeworks  opengl  openmirage  openscad  opensource  operational_transform  optimization  orchestration  orc_language  organization  organizing  os  osc  osdev  osl  osresearch  osx  owasp  p2p  packaging  painting  paper  paperman  papers  parallelism  parallelization  parinfer  parser  parsing  particles  patch  patents  pattern  patterns  paxos  pdf  performance  peripherals  perl  perspective  pgp  philosophy  photography  photoshop  physics  pinboard  pipe  pipeline  piperesearch  pipes  pixar  pixelart  pixiv  plan9  planning  platform  playlist  plugin  plugins  pmz  podcast  polycount  polygon  polygons  polymer.js  posix  post  postgres  postgresql  postmortem  powershell  practices  presentation  previz  printing  privacy  procedural  processes  production  productivity  programming  project  prolog  proof  proofs  propagator  protocol  protocols  psx  psyc  pulseaudio  pwa  pws  python  qmail  quaternion  query  queue  quickcheck  quote  q_language  racket  radio  ragel  rails  random  rant  raytracer  raytracing  reactjs  readability  rebol  reddit  redis  redlanguage  redo  refactoring  reference  regex  relational  reliablility  renderer  rendering  repl  replication  repo  repo:bitbucket  repo:github  repo:gitlab  repo:googlecode  repo:sourceforge  report  research  resource  rest  retopo  retrocomputing  retrogaming  review  rfc  riak  riemann  rigging  robotics  ros  rotoscoping  rpc  rtos  ruby  rule  rules  runtime  rust  r_language  safety  sakuga  sam  sample  scala  scalability  scaling  scenegraph  scheduler  schema  scheme  science  script  scripting  sculpting  sculptris  sdf  sdn  secondlife  security  sega  semantics  semanticweb  semver  serialization  server  service  servicediscovery  services  sexp  sgi  shaders  shading  sharing  shell  siggraph  signatures  signed-distance  simulation  sisal  sketchup  slides  slideshare  smalltalk  sml  snobol  society  sociology  softimage  software  software_development  solver  sorting  source  spec  specification  spir-v  spki  spreadsheet  spreadsheets  sprite  sprites  sql  sqlite  squeak  ssb  ssd  sse  ssh  ssl  stackoverflow  standards  startups  star_language  state  statecharts  statemachine  statement  steam  stellar_protocol  stop_motion  storage  story  storywriting  strangeloop  stream  streaming  streams  strings  style  subversion  summary  surf  svg  swift  sync  synchronization  syntax  synthesizer  sysadmin  system  systemd  tactics  talk  tcl  teaching  technique  technology  terminal  terminology  terrain  terra_language  testing  text  texture  textures  texture_mapping  texturing  text_processing  theory  threads  threejs  time  timeline  tip  tips  tla  to-get  to-read  to-watch  todo  tools  toot  toots  transactions  transcript  transducers  tree  tui  tumblr  tutorial  tutorials  tvpaint  tweet  tweets  twitter  types  typography  ubf  ui  uml  unhosted  unicode  unikernel  unity3d  unix  unreal  upspin  url  usability  uv  ux  vcs  vector  verification  verse  versioning  version_control  vfx  vgc  vi  via:popular  video  video:embedded  video:niconico  video:reddit  video:twitter  video:vimeo  video:youtube  videos  video_editing  video_editor  video_software  vim  virtualization  vision  visual  visualization  vm  voice  voxel  voxels  vpri  vr  vroid  vst  vsxu  vulkan  vvvv  w-hat  w3c  watches  wearable  web  webapp  webassembly  webcam  webclip  webdesign  webdev  webgl  webrtc  websockets  webvr  weird  wiki  wikipedia  win32  windows  wired  workflow  writing  x11  x86  xanadu  xen  xi_editor  xml  xmpp  xr  yang  zbrush  zeromq  zmq  zookeeper 

Copy this bookmark:



description:


tags: