POSTS

NATS 應用(一)

December 27, 2019

NATS 應用(一)

今天來介紹我對於NATS首次應用的方式與想法。

安裝測試使用Docker

 $ docker pull nats:latest

latest: Pulling from library/nats
f3c8f35bd9bc: Pull complete
4ba25f2430b4: Pull complete
Digest: sha256:83922f11aa2012176cc23401cc1a7648c6d0aeecd1597e04c04d1028908d155c
Status: Downloaded newer image for nats:latest
docker.io/library/nats:latest

執行與啟動

$ docker run \
  -d \
  -p 4222:4222 \
  -p 8222:8222 \
  --name=nats \
  nats:latest

eaad1e13140a8803a95c361b32b7d2c5aadf852bae50dfbfd0a8769aeaa551af

此時可以透過管理介面來了解狀態與相關資訊。

監看

$ go get github.com/nats-io/nats-top
$ nats-top

NATS server version 2.1.2 (uptime: 25m28s)
Server:
  Load: CPU:  1.0%  Memory: 10.8M  Slow Consumers: 0
  In:   Msgs: 0  Bytes: 0  Msgs/Sec: 0.0  Bytes/Sec: 0
  Out:  Msgs: 0  Bytes: 0  Msgs/Sec: 0.0  Bytes/Sec: 0

Connections Polled: 0
  HOST             CID    SUBS    PENDING     MSGS_TO     MSGS_FROM   BYTES_TO    BYTES_FROM  LANG     VERSION  UPTIME   LAST ACTIVITY

壓測

壓測在同一台筆電。

$ go get github.com/nats-io/nats.go
$ go install $GOPATH/src/github.com/nats-io/nats.go/examples/nats-bench/main.go
$ nats-bench -h
Usage: nats-bench [-s server (nats://127.0.0.1:4222)] [--tls] [-np NUM_PUBLISHERS] [-ns NUM_SUBSCRIBERS] [-n NUM_MSGS] [-ms MESSAGE_SIZE] [-csv csvfile] <subject>
  -creds string
    	User Credentials File
  -csv string
    	Save bench data to csv file
  -h	Show help message
  -ms int
    	Size of the message. (default 128)
  -n int
    	Number of Messages to Publish (default 100000)
  -np int
    	Number of Concurrent Publishers (default 1)
  -ns int
    	Number of Concurrent Subscribers
  -s string
    	The nats server URLs (separated by comma) (default "nats://127.0.0.1:4222")
  -tls
    	Use TLS Secure Connection

因為速度實在太快一開始測試使用100k,每則訊息128一下就跑完根本沒測到只好先從1m 1百萬開始測試的到 301,091 msgs/sec。

$ nats-bench -np 1 -n 1000000 -ms 128 foo
Starting benchmark [msgs=1000000, msgsize=128, pubs=1, subs=0]
Pub stats: 301,091 msgs/sec ~ 36.75 MB/sec

因為速度實在真的太快，有空來試試一百個發送端然後呢完全沒感受到壓力。的到的結果還更快Pub stats: 317,110 msgs/sec ~ 38.71 MB/sec

nats-bench -np 100 -n 1000000 -ms 128 foo
Starting benchmark [msgs=1000000, msgsize=128, pubs=100, subs=0]
Pub stats: 317,110 msgs/sec ~ 38.71 MB/sec
 [1] 75,883 msgs/sec ~ 9.26 MB/sec (10000 msgs)
 [2] 65,996 msgs/sec ~ 8.06 MB/sec (10000 msgs)
 [3] 56,034 msgs/sec ~ 6.84 MB/sec (10000 msgs)
 [4] 48,367 msgs/sec ~ 5.90 MB/sec (10000 msgs)
 [5] 40,431 msgs/sec ~ 4.94 MB/sec (10000 msgs)
 [6] 41,864 msgs/sec ~ 5.11 MB/sec (10000 msgs)
 [7] 43,557 msgs/sec ~ 5.32 MB/sec (10000 msgs)
 [8] 36,707 msgs/sec ~ 4.48 MB/sec (10000 msgs)
 [9] 35,855 msgs/sec ~ 4.38 MB/sec (10000 msgs)
 [10] 34,463 msgs/sec ~ 4.21 MB/sec (10000 msgs)
 [11] 35,862 msgs/sec ~ 4.38 MB/sec (10000 msgs)
 [12] 37,517 msgs/sec ~ 4.58 MB/sec (10000 msgs)
 [13] 41,028 msgs/sec ~ 5.01 MB/sec (10000 msgs)
 [14] 42,191 msgs/sec ~ 5.15 MB/sec (10000 msgs)
 [15] 39,630 msgs/sec ~ 4.84 MB/sec (10000 msgs)
 [16] 41,796 msgs/sec ~ 5.10 MB/sec (10000 msgs)
 [17] 42,467 msgs/sec ~ 5.18 MB/sec (10000 msgs)
 [18] 48,954 msgs/sec ~ 5.98 MB/sec (10000 msgs)
 [19] 43,827 msgs/sec ~ 5.35 MB/sec (10000 msgs)
 [20] 43,513 msgs/sec ~ 5.31 MB/sec (10000 msgs)
 [21] 44,559 msgs/sec ~ 5.44 MB/sec (10000 msgs)
 [22] 41,739 msgs/sec ~ 5.10 MB/sec (10000 msgs)
 [23] 39,829 msgs/sec ~ 4.86 MB/sec (10000 msgs)
 [24] 38,489 msgs/sec ~ 4.70 MB/sec (10000 msgs)
 [25] 40,094 msgs/sec ~ 4.89 MB/sec (10000 msgs)
 [26] 40,260 msgs/sec ~ 4.91 MB/sec (10000 msgs)
 [27] 44,679 msgs/sec ~ 5.45 MB/sec (10000 msgs)
 [28] 40,734 msgs/sec ~ 4.97 MB/sec (10000 msgs)
 [29] 38,506 msgs/sec ~ 4.70 MB/sec (10000 msgs)
 [30] 41,574 msgs/sec ~ 5.07 MB/sec (10000 msgs)
 [31] 41,218 msgs/sec ~ 5.03 MB/sec (10000 msgs)
 [32] 35,174 msgs/sec ~ 4.29 MB/sec (10000 msgs)
 [33] 38,259 msgs/sec ~ 4.67 MB/sec (10000 msgs)
 [34] 37,982 msgs/sec ~ 4.64 MB/sec (10000 msgs)
 [35] 41,917 msgs/sec ~ 5.12 MB/sec (10000 msgs)
 [36] 39,423 msgs/sec ~ 4.81 MB/sec (10000 msgs)
 [37] 41,626 msgs/sec ~ 5.08 MB/sec (10000 msgs)
 [38] 43,738 msgs/sec ~ 5.34 MB/sec (10000 msgs)
 [39] 50,766 msgs/sec ~ 6.20 MB/sec (10000 msgs)
 [40] 47,048 msgs/sec ~ 5.74 MB/sec (10000 msgs)
 [41] 46,145 msgs/sec ~ 5.63 MB/sec (10000 msgs)
 [42] 37,526 msgs/sec ~ 4.58 MB/sec (10000 msgs)
 [43] 38,351 msgs/sec ~ 4.68 MB/sec (10000 msgs)
 [44] 39,022 msgs/sec ~ 4.76 MB/sec (10000 msgs)
 [45] 40,510 msgs/sec ~ 4.95 MB/sec (10000 msgs)
 [46] 36,882 msgs/sec ~ 4.50 MB/sec (10000 msgs)
 [47] 34,149 msgs/sec ~ 4.17 MB/sec (10000 msgs)
 [48] 34,054 msgs/sec ~ 4.16 MB/sec (10000 msgs)
 [49] 33,852 msgs/sec ~ 4.13 MB/sec (10000 msgs)
 [50] 34,962 msgs/sec ~ 4.27 MB/sec (10000 msgs)
 [51] 36,692 msgs/sec ~ 4.48 MB/sec (10000 msgs)
 [52] 39,688 msgs/sec ~ 4.84 MB/sec (10000 msgs)
 [53] 42,418 msgs/sec ~ 5.18 MB/sec (10000 msgs)
 [54] 37,130 msgs/sec ~ 4.53 MB/sec (10000 msgs)
 [55] 39,705 msgs/sec ~ 4.85 MB/sec (10000 msgs)
 [56] 36,119 msgs/sec ~ 4.41 MB/sec (10000 msgs)
 [57] 36,564 msgs/sec ~ 4.46 MB/sec (10000 msgs)
 [58] 37,922 msgs/sec ~ 4.63 MB/sec (10000 msgs)
 [59] 33,952 msgs/sec ~ 4.14 MB/sec (10000 msgs)
 [60] 36,857 msgs/sec ~ 4.50 MB/sec (10000 msgs)
 [61] 33,471 msgs/sec ~ 4.09 MB/sec (10000 msgs)
 [62] 32,798 msgs/sec ~ 4.00 MB/sec (10000 msgs)
 [63] 37,200 msgs/sec ~ 4.54 MB/sec (10000 msgs)
 [64] 32,321 msgs/sec ~ 3.95 MB/sec (10000 msgs)
 [65] 37,842 msgs/sec ~ 4.62 MB/sec (10000 msgs)
 [66] 36,504 msgs/sec ~ 4.46 MB/sec (10000 msgs)
 [67] 34,631 msgs/sec ~ 4.23 MB/sec (10000 msgs)
 [68] 34,760 msgs/sec ~ 4.24 MB/sec (10000 msgs)
 [69] 34,743 msgs/sec ~ 4.24 MB/sec (10000 msgs)
 [70] 31,716 msgs/sec ~ 3.87 MB/sec (10000 msgs)
 [71] 33,829 msgs/sec ~ 4.13 MB/sec (10000 msgs)
 [72] 36,549 msgs/sec ~ 4.46 MB/sec (10000 msgs)
 [73] 34,890 msgs/sec ~ 4.26 MB/sec (10000 msgs)
 [74] 36,252 msgs/sec ~ 4.43 MB/sec (10000 msgs)
 [75] 39,384 msgs/sec ~ 4.81 MB/sec (10000 msgs)
 [76] 33,150 msgs/sec ~ 4.05 MB/sec (10000 msgs)
 [77] 34,656 msgs/sec ~ 4.23 MB/sec (10000 msgs)
 [78] 40,631 msgs/sec ~ 4.96 MB/sec (10000 msgs)
 [79] 40,569 msgs/sec ~ 4.95 MB/sec (10000 msgs)
 [80] 30,187 msgs/sec ~ 3.69 MB/sec (10000 msgs)
 [81] 37,223 msgs/sec ~ 4.54 MB/sec (10000 msgs)
 [82] 75,893 msgs/sec ~ 9.26 MB/sec (10000 msgs)
 [83] 36,271 msgs/sec ~ 4.43 MB/sec (10000 msgs)
 [84] 29,503 msgs/sec ~ 3.60 MB/sec (10000 msgs)
 [85] 35,251 msgs/sec ~ 4.30 MB/sec (10000 msgs)
 [86] 29,784 msgs/sec ~ 3.64 MB/sec (10000 msgs)
 [87] 24,450 msgs/sec ~ 2.98 MB/sec (10000 msgs)
 [88] 27,402 msgs/sec ~ 3.35 MB/sec (10000 msgs)
 [89] 27,374 msgs/sec ~ 3.34 MB/sec (10000 msgs)
 [90] 25,288 msgs/sec ~ 3.09 MB/sec (10000 msgs)
 [91] 24,921 msgs/sec ~ 3.04 MB/sec (10000 msgs)
 [92] 22,411 msgs/sec ~ 2.74 MB/sec (10000 msgs)
 [93] 22,599 msgs/sec ~ 2.76 MB/sec (10000 msgs)
 [94] 24,285 msgs/sec ~ 2.96 MB/sec (10000 msgs)
 [95] 33,144 msgs/sec ~ 4.05 MB/sec (10000 msgs)
 [96] 22,931 msgs/sec ~ 2.80 MB/sec (10000 msgs)
 [97] 34,064 msgs/sec ~ 4.16 MB/sec (10000 msgs)
 [98] 36,356 msgs/sec ~ 4.44 MB/sec (10000 msgs)
 [99] 39,388 msgs/sec ~ 4.81 MB/sec (10000 msgs)
 [100] 42,055 msgs/sec ~ 5.13 MB/sec (10000 msgs)
 min 22,411 | avg 38,327 | max 75,893 | stddev 8,480 msgs

然後我亂亂壓測期間觀看狀態最後的到資料記憶體最多只佔用24.8M!!(當然目前都只是一個訂閱所以就單純是發送速度而言)

NATS server version 2.1.2 (uptime: 55m18s)
Server:
  Load: CPU:  0.0%  Memory: 24.1M  Slow Consumers: 0
  In:   Msgs: 24.8M  Bytes: 3.1G  Msgs/Sec: 0.0  Bytes/Sec: 0
  Out:  Msgs: 0  Bytes: 0  Msgs/Sec: 0.0  Bytes/Sec: 0

Connections Polled: 0
  HOST             CID    SUBS    PENDING     MSGS_TO     MSGS_FROM   BYTES_TO    BYTES_FROM  LANG     VERSION  UPTIME   LAST ACTIVITY

來看看比較像是可能的實際狀況，五個發佈的五個訂閱的。數據發佈: 119,294 msgs/sec ~ 14.56 MB/sec 數據訂閱: 526,331 msgs/sec ~ 64.25 MB/sec

 $ nats-bench -np 5 -ns 5 -n 1000000 -ms 128 foo
Starting benchmark [msgs=1000000, msgsize=128, pubs=5, subs=5]
NATS Pub/Sub stats: 631,161 msgs/sec ~ 77.05 MB/sec
 Pub stats: 119,294 msgs/sec ~ 14.56 MB/sec
  [1] 26,516 msgs/sec ~ 3.24 MB/sec (200000 msgs)
  [2] 25,106 msgs/sec ~ 3.06 MB/sec (200000 msgs)
  [3] 25,298 msgs/sec ~ 3.09 MB/sec (200000 msgs)
  [4] 24,996 msgs/sec ~ 3.05 MB/sec (200000 msgs)
  [5] 23,858 msgs/sec ~ 2.91 MB/sec (200000 msgs)
  min 23,858 | avg 25,154 | max 26,516 | stddev 846 msgs
 Sub stats: 526,331 msgs/sec ~ 64.25 MB/sec
  [1] 105,774 msgs/sec ~ 12.91 MB/sec (1000000 msgs)
  [2] 105,432 msgs/sec ~ 12.87 MB/sec (1000000 msgs)
  [3] 105,320 msgs/sec ~ 12.86 MB/sec (1000000 msgs)
  [4] 105,313 msgs/sec ~ 12.86 MB/sec (1000000 msgs)
  [5] 105,269 msgs/sec ~ 12.85 MB/sec (1000000 msgs)
  min 105,269 | avg 105,421 | max 105,774 | stddev 184 msgs

實際驗證

我們開了一個全新的Laravel 專案，只在頁面做一個很笨的查詢。

 $ wrk -c 10 -d 30s -t 1  http://127.0.0.1:8000/
Running 30s test @ http://127.0.0.1:8000/
  1 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   138.54ms   34.64ms 284.19ms   82.26%
    Req/Sec    64.87     12.60    80.00     75.34%
  1939 requests in 30.08s, 7.51MB read
  Socket errors: connect 0, read 1939, write 0, timeout 0
Requests/sec:     64.45
Transfer/sec:    255.55KB

這邊可以發現在30秒內壓測連線數飆到將近三百！！

這邊我們改了一個 NATS接收端，程式碼也是來做相同的事。

Laravel這邊變成發送請求，與接收結果。

這樣的方式類似RPC call!沒錯這邊先以這方式說明好處。

然後再做相同的壓測！

 $ wrk -c 10 -d 30s -t 1  http://127.0.0.1:8000/
Running 30s test @ http://127.0.0.1:8000/
  1 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   130.00ms   34.69ms 285.86ms   82.18%
    Req/Sec    66.04     16.15    90.00     88.47%
  1964 requests in 30.10s, 7.61MB read
  Socket errors: connect 0, read 1964, write 0, timeout 0
Requests/sec:     65.26
Transfer/sec:    259.06KB

各位可以看到速度並沒有差異！

但是！！連線數只有4!!與原本的將近三百有很大差異！

結論

我們以類似RPC CALL方式驗證，將db工作給另外後端的服務去執行。這樣有哪些好處呢？大家都知道MySQL連線術士有限制的依照ram大小！所以你用雲端主機像是aws你要選很大機器才能應付這樣少少流量，這樣不是很吃虧？！

所以光這部份可以替你省下不少錢！這幾年較少去探討程式因為php已經沒有太多可以發揮探討的部分。大部分我卓重於程式整合與架構應用，以後會多多介紹這方面！

NATS
Laravel

NATS 應用(一)

安裝 測試使用Docker

執行與啟動

監看

壓測

實際驗證

結論

安裝測試使用Docker