Merge pull request #83 from lukaszbudnik/ping-reachability-analyzer

implemented ping and reachability analyzer functionality
2023-11-15 20:13:48 +01:00
parent 3ccac4010f f6cd6893fd
commit bfb0a19737
8 changed files with 312 additions and 24 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
 debug.test
+cover.out
 yosoy
--- a/4
+++ b/4
@@ -2,8 +2,12 @@ FROM golang:1.21-alpine as builder

 LABEL maintainer="Łukasz Budnik lukasz.budnik@gmail.com"

+# install prerequisites
+RUN apk update && apk add git
+
 # build yosoy
 ADD . /go/yosoy
+RUN go env -w GOPROXY=direct
 RUN cd /go/yosoy && go build

 FROM alpine:3.18
--- a/README.md
+++ b/README.md
@@ -1,15 +1,18 @@
 # yosoy ![Go](https://github.com/lukaszbudnik/yosoy/workflows/Go/badge.svg) ![Docker](https://github.com/lukaszbudnik/yosoy/workflows/Docker%20Image%20CI/badge.svg)

-yosoy is a HTTP service for stubbing and prototyping distributed applications. It is a service which will introduce itself to the caller and print some useful information about its environment. "Yo soy" in español means "I am".
+yosoy is an HTTP service for stubbing and prototyping distributed applications. It is a service that introduces itself to the caller and prints useful information about its runtime environment. 

-yosoy is extremely useful when creating a distributed application stub and you need to see more meaningful responses than a default nginx welcome page.
+yosoy is extremely useful when creating a stub for a distributed application, as it provides more meaningful responses than, for example, a default nginx welcome page. Further, yosoy incorporates a built-in reachability analyzer to facilitate troubleshooting connectivity issues in distributed systems. A dedicated reachability analyzer endpoint validates network connectivity between yosoy and remote endpoints.

 Typical use cases include:

- testing HTTP routing & ingress
- testing HTTP load balancing
- testing HTTP caching
- stubbing and prototyping distributed applications
+- Testing HTTP routing and ingress
+- Testing HTTP load balancing
+- Testing HTTP caching
+- Executing reachability analysis
+- Stubbing and prototyping distributed applications
+
+"Yo soy" means "I am" in Spanish.

 ## API

@@ -31,19 +34,16 @@ yosoy responds to all requests with a JSON containing the information about:
  - Env variables if `YOSOY_SHOW_ENVS` is set to `true`, `yes`, `on`, or `1`
  - Files' contents if `YOSOY_SHOW_FILES` is set to a comma-separated list of (valid) files

-Checkout out [Sample JSON response](#sample-json-response) below to see how useful yosoy is when troubleshooting/stubbing/prototyping distributed applications.
+Check [Sample JSON response](#sample-json-response) to see how you can use yosoy for stubbing/prototyping/troubleshooting distributed applications.
+
+Check [ping/reachability analyzer](#pingreachability-analyzer) to see how you can use yosoy for troubleshooting network connectivity.

 ## Docker image

-The docker image is available on docker hub:
+The docker image is available on docker hub and ghcr.io:

 ```sh
 docker pull lukasz/yosoy
-```
-
-and ghcr.io:
-
-```sh
 docker pull ghcr.io/lukaszbudnik/yosoy
 ```

@@ -51,7 +51,7 @@ It exposes HTTP service on port 80.

 ## Kubernetes example

-There is a sample Kubernetes deployment file in the `test` folder. It uses both `YOSOY_SHOW_ENVS` and `YOSOY_SHOW_FILES`. The deployment uses Kubernetes Downward API to expose labels and annotations as volume files which are then returned by yosoy.
+There is a sample Kubernetes deployment file in the `test` folder. It uses both `YOSOY_SHOW_ENVS` and `YOSOY_SHOW_FILES` features. The deployment uses Kubernetes Downward API to expose labels and annotations as volume files which are then returned by yosoy.

 Deploy it to minikube and execute curl to the service a couple of times:

@@ -131,3 +131,42 @@ A sample yosoy JSON response to a request made from a single page application (S
  }
 }
 ```
+
+## ping/reachability analyzer
+
+yosoy includes a simple ping/reachability analyzer. You can use this functionality when prototyping distributed systems to validate whether a given component can reach a specific endpoint. yosoy exposes a dedicated `/_/yosoy/ping` endpoint which accepts the following 3 query parameters:
+
+* `h` - required - hostname of the endpoint
+* `p` - required - port of the endpoint
+* `n` - optional - network, all valid Go networks are supported (including the most popular ones like `tcp`, `udp`, IPv4, IPV6, etc.). If `n` parameter is not provided, it defaults to `tcp`. Go will throw an error if `n` parameter will be set to unknown network.
+
+For example, to test if yosoy can connect to `google.com` on port `443` using default `tcp` network use the following command:
+
+```bash
+curl "$URL/_/yosoy/ping?h=google.com&p=443"
+```
+
+To see an unsuccessful response you may use localhost with some random port number:
+
+```bash
+curl "$URL/_/yosoy/ping?h=127.0.0.1&p=12345"
+```
+
+## Building and testing locally
+
+Here are some commands to get you started.
+
+Run yosoy directly on port 80.
+
+```bash
+go test -coverprofile cover.out
+go tool cover -html=cover.out
+go run server.go
+```
+
+Building local Docker container and run it on port 8080:
+
+```bash
+docker build -t yosoy-local:latest .
+docker run --rm --name yosoy-local -p 8080:80 yosoy-local:latest
+```
--- a/go.mod
+++ b/go.mod
@@ -1,9 +1,16 @@
 module github.com/lukaszbudnik/yosoy

-go 1.16
+go 1.21

 require (
 	github.com/gorilla/handlers v1.5.2
 	github.com/gorilla/mux v1.8.1
 	github.com/stretchr/testify v1.8.4
 )
+
+require (
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/felixge/httpsnoop v1.0.3 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)
--- a/go.sum
+++ b/go.sum
@@ -1,4 +1,3 @@
-github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/felixge/httpsnoop v1.0.3 h1:s/nj+GCswXYzN5v2DpNMuMQYe+0DDwt5WVCU6CWBdXk=
@@ -9,15 +8,9 @@ github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
 github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
-github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
-github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
--- a/server.go
+++ b/server.go
@@ -3,6 +3,7 @@ package main
 import (
 	"encoding/json"
 	"log"
+	"net"
 	"net/http"
 	"os"
 	"strings"
@@ -26,6 +27,14 @@ type response struct {
 	Files        map[string]string   `json:"files,omitempty"`
 }

+type errorResponse struct {
+	Error string `json:"error"`
+}
+
+type successResponse struct {
+	Message string `json:"message"`
+}
+
 var counter = 0
 var hostname = os.Getenv("HOSTNAME")

@@ -85,6 +94,55 @@ func handler(w http.ResponseWriter, req *http.Request) {
 	json.NewEncoder(w).Encode(response)
 }

+func ping(w http.ResponseWriter, req *http.Request) {
+	// get h, p, t parameters from query string
+	hostname := req.URL.Query().Get("h")
+	port := req.URL.Query().Get("p")
+	network := req.URL.Query().Get("n")
+
+	// return HTTP BadRequest when hostname is empty
+	if hostname == "" {
+		w.WriteHeader(http.StatusBadRequest)
+		w.Header().Add("Content-Type", "application/json; charset=utf-8")
+		json.NewEncoder(w).Encode(&errorResponse{"hostname is empty"})
+		return
+	}
+	// return HTTP BadRequest when port is empty
+	if port == "" {
+		w.WriteHeader(http.StatusBadRequest)
+		w.Header().Add("Content-Type", "application/json; charset=utf-8")
+		json.NewEncoder(w).Encode(&errorResponse{"port is empty"})
+		return
+	}
+	// if network is empty set default to tcp
+	if network == "" {
+		network = "tcp"
+	}
+	// ping the hostname and port by opening a socket
+	err := pingHost(hostname, port, network)
+	if err != nil {
+		w.WriteHeader(http.StatusInternalServerError)
+		w.Header().Add("Content-Type", "application/json; charset=utf-8")
+		json.NewEncoder(w).Encode(&errorResponse{err.Error()})
+		return
+	}
+	// return HTTP OK
+	w.WriteHeader(http.StatusOK)
+	w.Header().Add("Content-Type", "application/json; charset=utf-8")
+	json.NewEncoder(w).Encode(&successResponse{"ping succeeded"})
+}
+
+func pingHost(hostname, port, network string) error {
+	// open a socket to hostname and port
+	conn, err := net.Dial(network, hostname+":"+port)
+	if err != nil {
+		return err
+	}
+	// close the socket
+	conn.Close()
+	return nil
+}
+
 func remoteAddrWithoutPort(req *http.Request) string {
 	remoteAddr := req.RemoteAddr
 	if index := strings.LastIndex(remoteAddr, ":"); index > 0 {
@@ -99,6 +157,7 @@ func main() {
 	r := mux.NewRouter()

 	r.Handle("/favicon.ico", r.NotFoundHandler)
+	r.HandleFunc("/_/yosoy/ping", ping).Methods(http.MethodGet)
 	r.PathPrefix("/").HandlerFunc(preflight).Methods(http.MethodOptions)
 	r.PathPrefix("/").HandlerFunc(handler).Methods(http.MethodGet, http.MethodPut, http.MethodPatch, http.MethodPost, http.MethodDelete, http.MethodConnect, http.MethodHead, http.MethodTrace)

--- a/server_test.go
+++ b/server_test.go
@@ -3,6 +3,7 @@ package main
 import (
 	"bytes"
 	"encoding/json"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"os"
@@ -13,7 +14,7 @@ import (

 func TestHandler(t *testing.T) {
 	os.Setenv("YOSOY_SHOW_ENVS", "true")
-	os.Setenv("YOSOY_SHOW_FILES", ".gitignore")
+	os.Setenv("YOSOY_SHOW_FILES", ".gitignore,/file/does/not/exist")

 	req, err := http.NewRequest("GET", "https://example.org/sample/path?one=jeden&two=dwa", nil)
 	if err != nil {
@@ -50,3 +51,187 @@ func TestHandler(t *testing.T) {
 	// test cors
 	assert.Contains(t, result.Header["Access-Control-Allow-Origin"], "*")
 }
+
+// write test for request /_/yosoy/ping without any query parameters, the request should return bad request 400 error and return JSON error about missing hostname parameter
+func TestHandlerPingNoParameters(t *testing.T) {
+	req, err := http.NewRequest("GET", "https://example.org/_/yosoy/ping", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(ping)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusBadRequest {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusBadRequest)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+
+	var response errorResponse
+	json.Unmarshal(buf.Bytes(), &response)
+
+	// test response
+	assert.Equal(t, "hostname is empty", response.Error)
+}
+
+// write test for request /_/yosoy/ping with h parameter, the request should return bad request 400 error and return JSON error about port is empty
+func TestHandlerPingWithHostname(t *testing.T) {
+	req, err := http.NewRequest("GET", "https://example.org/_/yosoy/ping?h=example.org", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(ping)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusBadRequest {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusBadRequest)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+
+	var response errorResponse
+	json.Unmarshal(buf.Bytes(), &response)
+
+	// test response
+	assert.Equal(t, "port is empty", response.Error)
+}
+
+// write test for request /_/yosoy/ping with h=127.0.0.1 parameter and p=8123 parameter, the request should return bad request 400 error and return JSON error about tcp connection issue
+func TestHandlerPingWithHostnameAndPort(t *testing.T) {
+	req, err := http.NewRequest("GET", "https://example.org/_/yosoy/ping?h=127.0.0.1&p=8123", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(ping)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusInternalServerError {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusInternalServerError)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+
+	var response errorResponse
+	json.Unmarshal(buf.Bytes(), &response)
+
+	// test response
+	assert.Equal(t, "dial tcp 127.0.0.1:8123: connect: connection refused", response.Error)
+}
+
+// write test for request /_/yosoy/ping with h=127.0.0.1 parameter and p=8123 parameter, the request should return 200 ok and return JSON with message ping succeeded
+func TestHandlerPingWithHostnameAndPortSuccess(t *testing.T) {
+
+	// create tcp process to listen on port 8123
+	listener, err := net.Listen("tcp", "localhost:8123")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer listener.Close()
+
+	req, err := http.NewRequest("GET", "https://example.org/_/yosoy/ping?h=127.0.0.1&p=8123", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(ping)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusOK {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusOK)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+
+	var response successResponse
+	json.Unmarshal(buf.Bytes(), &response)
+
+	// test response
+	assert.Equal(t, "ping succeeded", response.Message)
+}
+
+// write test for request /_/yosoy/ping?h=127.0.0.1&p=8123&n=qwq, the request should return 500 internal server error and return JSON with error "dial qwq: unknown network qwq"
+func TestHandlerPingWithHostnameAndPortAndNetwork(t *testing.T) {
+
+	// create tcp process to listen on port 8123
+	listener, err := net.Listen("tcp", "localhost:8123")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer listener.Close()
+
+	req, err := http.NewRequest("GET", "https://example.org/_/yosoy/ping?h=127.0.0.1&p=8123&n=qwq", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(ping)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusInternalServerError {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusInternalServerError)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+	var response errorResponse
+	json.Unmarshal(buf.Bytes(), &response)
+	// test response
+	assert.Equal(t, "dial qwq: unknown network qwq", response.Error)
+}
+
+// write test for preflight HTTP Options request, verify that all headers are set
+func TestHandlerPreflight(t *testing.T) {
+	req, err := http.NewRequest("OPTIONS", "https://example.org/test", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	req.Header.Set("Accept", "*/*")
+
+	rr := httptest.NewRecorder()
+	handler := http.HandlerFunc(preflight)
+
+	handler.ServeHTTP(rr, req)
+
+	if status := rr.Code; status != http.StatusOK {
+		t.Errorf("handler returned wrong status code: got %v want %v", status, http.StatusOK)
+	}
+
+	result := rr.Result()
+	buf := new(bytes.Buffer)
+	buf.ReadFrom(result.Body)
+
+	// test response
+	assert.Equal(t, "*", result.Header.Get("Access-Control-Allow-Origin"))
+	assert.Equal(t, "*", result.Header.Get("Access-Control-Allow-Methods"))
+	assert.Equal(t, "*", result.Header.Get("Access-Control-Allow-Headers"))
+	assert.Equal(t, "true", result.Header.Get("Access-Control-Allow-Credentials"))
+	assert.Equal(t, "600", result.Header.Get("Access-Control-Max-Age"))
+	assert.Equal(t, "*", result.Header.Get("Access-Control-Expose-Headers"))
+}
--- a/test/deployment.yaml
+++ b/test/deployment.yaml
@@ -16,7 +16,7 @@ spec:
    spec:
      containers:
      - name: yosoy
-        image: lukasz/yosoy:2.0.3
+        image: lukasz/yosoy:edge
        env:
          - name: YOSOY_SHOW_ENVS
            value: "true"