Schedulable lightweight virtual processes and concurrent loops P&L: -11.5 (≃ -982 CNY)

An approach to building concurrent software
YAML 項目 產品

Imagine a nested for loop, it is composed of multiple loops. We can simulate concurrency by executing parts of loops and rapidly switch between them.

This is what a kernel scheduler does and I think it's a very interesting part of an operating system. Unfortunately most programs are not written to be multithreaded so computers use very little of resources available to them.


年度 內部收益率: -1.0000
最新 淨現值@貼現率=0.1: -11.4888 ħ (-981.23 CNY)
(別通知) 請,登錄

(別通知) 請,登錄

(別通知) 請,登錄

(別通知) 請,登錄

(別通知) 請,登錄

(別通知) 請,登錄

(別通知) (可選) 請,登錄

我意識到我的邏輯錯誤。 線程 0 負責循環迭代 0-8 線程1負責8-16 線程2負責16-24 等等 由於不處理環繞的數學邏輯,我目前正在錯誤的線程上執行循環迭代。我使用模運算符。 理想情況下,每個股票只在線程上調用其範圍。

I realised my bug in my logic. Thread 0 is responsible for loop iterations 0-8 Thread 1 is responsible for 8-16 Thread 2 is responsible for 16-24 And so on I'm currently executing loop iterations on the wrong thread due to a mathematical logic of not handling wraparound. I use the modulo operator. Ideally each ticker is only called on the thread for its range.

(別通知) 請,登錄

我有一個問題需要解決。這與並行性無關。

想象一下,我有一個看起來像這樣的嵌套循環

``` 對於信中的信件:

對於數字中的數字:

打印(字母編號)

對於符號中的符號:

打印(字母符號) ```

我希望這三個循環同時運行。 一種方法是根據索引選擇要運行的函數!並將數字和字母循環合併在一起。 所以我們得到

A1

A2

這種方法的問題在於循環不是分開的。它們是一個已合併的循環。

我想我可以通過使集合成爲一個多集合來解決這個問題。 [字母、[數字、符號] 並從每個子列表循環中挑選並將循環作爲單獨的對象公開。

I have a problem I need to solve. Which isn't related to the parallellism.

Imagine I have a nested loop that looks like this

``` For letter In letters:

For number in numbers:

Print(letter + number)

For symbol in symbols:

Print(letter + symbol) ```

I want these three loops to run concurrently. One approach is to pick the function to run based on the indexes! And merge the numbers and letters together round robin. So we get

A1

A2

The problem with this approach is that the loops aren't separate. They're one loop that has been merged.

I think I can solve this by causing collections to be a multi collection. [Letters, [numbers, symbols] And picking from each sublist round robin and exposing the loops as separate objects.

(別通知) 請,登錄

我創建了一個沒有 Joinable 循環的多線程版本。但此時我正在休息。 我需要以某種方式將多線程與可連接循環聯繫起來。 當 Joinable 循環收到 2 個或更多值時,它會繼續處理。這允許在管道處理中進行拆分和合並。 我希望加入的進程儘快發生並且是多線程的。 在我的設計中——我還沒有在 Java 中實現——每個線程都有一個調度程序,它重新檢查可以勾選的內容並勾選它。 問題是在線程上拆分滴答聲,如果只有一層代碼,這很容易。 您一次只能勾選 8 個批次。在單獨的線程中。 我的示例有一個包含 3 個集合的嵌套循環,並拆分爲兩個任務,然後這兩個任務合併爲一個任務以打印出結果。 我希望分批執行嵌套循環、單獨任務和連接任務。 需要一種將併發循環發送到舊線程的方法。可以有一個稱爲滴答池的共享集合,由所有線程檢查。 當前產品編號與線程相關聯。 我選擇 ÷ 8 是因爲 64 是 8 的倍數。清理每個線程的批次數。

```

雖然(真){

報價池中的 For 循環:

如果 current[loop] < thisThreadN[loop]:

當前[循環] = 當前[循環] 1

NewTickersOrResult = Loop.tick()

如果 NewTickersOrResult.isTickers:

對於 NewTickersOrResult.tickers 中的 newTicker:

 Current[loop] = threadNum × newTicker.size() / threadCount

 thisThreadN[loop] = threadNum × newTicker.size() / threadCount threadCount

Pool.extend(NewTickersOrResult.tickers)

別的:

打印(NewTickersOrResult.result)

}

```

I created a multithreading version of this without the Joinable loops. But at this time I am taking a break. I need to somehow tie the multithreading to the joinable loop. When Joinable loop received 2 or more values, it continues processing. This allows a split and merge in pipeline processing. I want joined processes to occur as soon as they can and to be multithreaded. In my design - that I am yet to implement in Java - each thread has a scheduler that rechecks what can be ticked and ticks it. The problem is splitting ticks over threads, it's easy if there is only one level of tickers. You can just tick batches of 8 at a time. In separate threads. My example has a nested loop of 3 collections and a split into two tasks and then those two tasks join into one task to print out the results. I want the nested loops, separate tasks and the joined tasks to be executed in batches. Kind of need a way to send concurrent loops to old threads. Could have a shared collection called the tick pool which is checked by all threads. The current product number is associated with a thread. I picked ÷ 8 due to the 64 being a multiple of 8. Clean number of batches per thread.

```

While (true) {

For loop in tick pool:

If current[loop] < thisThreadN[loop]:

current[loop] = current[loop] + 1

NewTickersOrResult = Loop.tick()

If NewTickersOrResult.isTickers:

For newTicker in NewTickersOrResult.tickers:

 Current[loop] = threadNum × newTicker.size() / threadCount

 thisThreadN[loop] = threadNum × newTicker.size() / threadCount + threadCount

Pool.extend(NewTickersOrResult.tickers)

Else:

Print(NewTickersOrResult.result)

}

```

(別通知) 請,登錄

我正在考慮如何使用多個 CPU 內核的性能。它需要對我們如何編寫代碼進行徹底的重新思考!

我突然想到,循環可以與我的設計簡單地並行化。

N = 0 .. N = 3×3×3

如果您在每個 N 的所有線程上運行 tick 方法,您可以一次運行整個循環。

I am thinking of how to use the performance of multiple CPU cores. It requires a drastic rethinking of how we write code!

It occurred to me that loops could be trivially parallelized with my design.

N = 0 .. N = 3×3×3

If you ran the tick method on all threads with every N, you could run the entire loop in one go.

(別通知) 請,登錄

我需要編寫一個 Joinable Ticker 來等待來自多個代碼的輸入,然後再發送輸出。 這讓我們可以創建拆分和合並的管道。

I need to write a Joinable Ticker that waits for inputs from multiple tickers before sending output along. This lets us create pipelines that split and merge.

(別通知) 請,登錄

我要補充一點,每個線程只需要一個 while (true) 循環。 其他一切都可以在一個線程中同時調度。

I shall add that you only need one while (true) loop per thread. Everything else can be concurrently scheduled within a thread.



    :  -- 
    : Mindey
    :  -- 
    

chronological,
(別通知) 請,登錄

這就是我稱之爲虛擬併發的原因。 您需要使用線程來提供 IO 併發 任何永遠循環或阻塞的東西都不能組合。因此,我將編寫我所有的代碼,以嘗試從不阻塞和重入。這是我在開發多線程調度程序時學到的重要一課。

阻塞對程序是不可見的。該程序不知道它正在阻塞。您需要知道方法可以阻止以解決它。 我的另一個想法是並行 while do 循環,它通過語法糖將阻塞變爲非阻塞。 看起來像這樣

A1 = 並行同時 { Buf = socket.recv(1024) } 做 { 並行 for (Socket socket : sockets) { Socket.send(buf) } } 交叉合併 A1

此語法啓動多個線程以阻止接收數據。對於每一個,它都會在線程池中啓動一個線程來處理它並將其並行發送到每個連接。

我的另一個想法是更改優先執行順序。該代碼中的調度程序每次都以相同的順序輪詢代碼。沒有理由我們不能比其他人更多地執行循環。

That's why I call it virtual concurrency. You would need to use threads to provide IO concurrency Anything that loops forever or blocks cannot be composed. So I shall write all my code going forward to try never block and be reetrant. This is an important lesson I learned while developing my multithreaded scheduler.

Blocking is invisible to the program. The program isn't aware it is blocking. You need to know that a method can block to work around it. My other idea is a parallel while do loop which changes blocking to be non blocking through syntactic sugar. It looks like this

A1 = parallel while { Buf = socket.recv(1024) } Do { Parallel for (Socket socket : sockets) { Socket.send(buf) } } Crossmerge A1

This syntax spins up multiple threads to block on receiving data. And for each one it spins up a thread in a thread pool to handle it and send it to every connection in parallel.

Another idea I have is to change the priority execution order. That scheduler in that code round robins the tickers in the same order every time. There's no reason why we cannot execute the loops some more than others.



    : Mindey
    :  -- 
    :  -- 
    

chronological,
(別通知) 請,登錄

有趣的。你沒有使用任何原生支持 concurrency in Python,並且只使用了基本的枚舉、索引、賦值。我想,這就是理解所需要的。這具有教育價值。

Interesting. You did not use any native support for concurrency in Python, and used only basic enumeration, indexing, assignment. I guess, that is what it takes to understand. This has educational value.

(別通知) 請,登錄