diff options
| author | D Delmar Davis <don@suspectdevices.com> | 2024-05-05 14:04:05 -0700 |
|---|---|---|
| committer | D Delmar Davis <don@suspectdevices.com> | 2024-05-05 14:04:05 -0700 |
| commit | cbdcb7dae62d7962a2d97c6086e47ed2c9ce46a5 (patch) | |
| tree | 32dc11bd9bbc95c22b07ff9cb4fbe7a558799c84 /docs/serverdocs/OperationsGuide.md | |
| parent | 8a6131046187f9ae407a315a2d5dc1dd545f07e0 (diff) | |
Update the netplan examples
Diffstat (limited to 'docs/serverdocs/OperationsGuide.md')
| -rw-r--r-- | docs/serverdocs/OperationsGuide.md | 206 |
1 files changed, 150 insertions, 56 deletions
diff --git a/docs/serverdocs/OperationsGuide.md b/docs/serverdocs/OperationsGuide.md index 6864397..cff4f67 100644 --- a/docs/serverdocs/OperationsGuide.md +++ b/docs/serverdocs/OperationsGuide.md @@ -1,29 +1,29 @@ -<!-- OperationsGuide, Version: 28, Modified: 2020/12/02, Author: feurig --> -# Server Modernization +<!-- OperationsGuide, Version: 28, Modified: 2020/12/02, Author: feurig -->
+# Server Modernization
## Overview
- +
- +
### Phase I
- +
Phase one of the server modernization shifted away from multipurposed servers and kvms to lxc/lxd based containers.
- +
* Moving all legacy system functions onto separate linux containers isolated from each other.
* Use mirrored disk systems to insure that disk corruption does not lead to data corruption.
* Start giving a shit about the systems, code, and sites on them.
* Own your code/data. (If your free code hosting system is shutdown or taken over by Microsoft is it really free)
- +
### Server Modernization Phase II
- +
Phase two extends on this by integrate Ansible into system maintenance tasks.
- +
* Integrate Ansible into system maintenance tasks
* Reevaluate Centos and other RPM based containers built using playbooks vs profiles/scripts/cloud-init _while maintaining current security model_
* Develop off site backup strategy.
* Clean up the cruft (If it doesn't bring you joy DTMFA)
- +
### SMP III _Make Shit Happen / Own Your Shit_
- +
* Work on secure and efficient traffic in and out of home lans (Privoxy,DNS based ad blocking,squid etc)
* Continue to refine server operation/maintanance.
* Build Gitlab and other alternatives to trac/git and evaluate workflows.
@@ -32,38 +32,40 @@ Phase two extends on this by integrate Ansible into system maintenance tasks. * Start new projects.
* Distribute data and backups over the network to home servers.
* [Document home server/network setup](edge-server-configuration/)
- - +
+
### Goals.
- +
* Security
* Flexibility
* Simplification
- +
### Isolation
- +
* network
* performance
* disk
- +
## Hardware
- +
At present the environment contains a vpn capable router (Knight) and two enterrise class servers
- -* bs2020 , a Dell PowerEdge R610 [[br]]and
+
+* aoc2024 , a Dell PowerEdge R610 [[br]]and
* kb2018 a HP ProLiant DL380 (g7) .
## Network
- +
The network is divided into 3 segments
- +
* 192.168.31.0/24 a private administrative lan
* tbd.tbd.tbd.tbd/? a private vpn for home offices
* 198.202.31.129/25 A public facing lan.
The hosts themselves do not have any public facing interfaces and are only accessible though the admin lan. The containers which handle all public facing work do so via an anonymous bridge configuration, allowing them to access the internet directly without allowing external access to the servers.
- -| | | | | bs2020 ports|
+
+### AOC network config
+
+| | | | | aoc2024 ports|
|---|---|---|---|-----------------|
|port| Interface| IP Address/mask | linux device| purpose|
| 1 | eno1 | 192.168.31.158/24 | eno1 |internal / admin lan|
@@ -71,12 +73,58 @@ The hosts themselves do not have any public facing interfaces and are only acces | 3 | br1 | 0.0.0.0/0 | eno3 |Public Interface for infrastructure servers|
| 4 | br0 | 0.0.0.0/0 | eno4 |Public Interface for dev/deploymant servers|
| idrac | | 192.168.31.121/24 | |remote console|
- -As Drawn|As Deployed. ----|--- - |  - +As Drawn|As Deployed.
+---|---
+ | 
+
+/etc/netplan/00-allthethings.yml
+```
+network:
+ ethernets:
+ eno1:
+ dhcp4: false
+ dhcp6: false
+ eno2:
+ dhcp4: false
+ dhcp6: false
+ eno3:
+ dhcp4: false
+ dhcp6: false
+ eno4:
+ dhcp4: false
+ dhcp6: false
+ bridges:
+ br1:
+ dhcp4: false
+ dhcp6: false
+ interfaces:
+ - eno4
+ br0:
+ dhcp4: false
+ dhcp6: false
+ interfaces:
+ - eno3
+ br3:
+ dhcp4: false
+ dhcp6: false
+ addresses:
+ - 192.168.31.158/24
+ routes:
+ - to: default
+ via: 192.168.31.1
+ nameservers:
+ addresses:
+ - 198.202.31.132
+ - 198.202.31.141
+ search: [suspectdevices.com fromhell.com vpn]
+ interfaces:
+ - eno1
+
+ version: 2
+```
+
+### KB2018 Network Config
| | | | | kb2018 ports|
|---|---|---|---|-----------------|
@@ -85,13 +133,59 @@ As Drawn|As Deployed. | 3 | enp4s0f0 | ?.?.?.?/?? | enp4s0f0 | vpn for home/office networks |
| 2 | br1 | 0.0.0.0/32 | enp3s0f1 |Public Interface for infrastructure servers|
| 1 | br0 | 0.0.0.0/32 | enp3s0f0 |Public Interface for dev/deploymant servers|
-| ilo | | 192.168.31.119/24 | |remote console| +| ilo | | 192.168.31.119/24 | |remote console|
+
+As Drawn|As Deployed.
+---|---
+ | 
+
+#### As implimented in /etc/netplan/xxxxx
+
+```
+network:
+ ethernets:
+ enp3s0f0:
+ dhcp4: false
+ dhcp6: false
+ enp3s0f1:
+ dhcp4: false
+ dhcp6: false
+ enp4s0f0:
+ dhcp4: false
+ dhcp6: false
+ enp4s0f1:
+ dhcp4: false
+ dhcp6: false
+ bridges:
+ br1:
+ dhcp4: false
+ dhcp6: false
+ interfaces:
+ - enp4s0f1:
+ br0:
+ dhcp4: false
+ dhcp6: false
+ interfaces:
+ - enp4s0f0:
+ br3:
+ dhcp4: false
+ dhcp6: false
+ addresses:
+ - 192.168.31.159/24
+ routes:
+ - to: default
+ via: 192.168.31.1
+ nameservers:
+ addresses:
+ - 198.202.31.132
+ - 198.202.31.141
+ search: [suspectdevices.com fromhell.com vpn]
+ interfaces:
+ - enp4s0f1
+
+ version: 2
+```
-As Drawn|As Deployed. ----|--- - |  - - See: [https://bitbucket.org/suspectdevicesadmin/ansible/src/master/hosts](https://bitbucket.org/suspectdevicesadmin/ansible/src/master/hosts) which is built referencing [a google doc with proposed allocations](https://docs.google.com/spreadsheets/d/1KRkqdYvgRtV4vu6AGzdLWJVGTIsV2o2iSSJBEFMZJAw/edit#gid=0)
## Server OS, Filesystems and Disk layout
@@ -99,7 +193,7 @@ The servers are both running a standard install Ubuntu Server LTS, along with th ### Disk Layout
The system disks on both servers use hardware raid 1+0 mirroring. The containers are able to take advantage of zfs mirroring and caching.
- +
| | | | | | bs2020 disks|
|---|---|---|---|---|-----------------|
|disk|device/pool | bay | type|mount point(s)|purpose/notes|
@@ -114,7 +208,7 @@ The system disks on both servers use hardware raid 1+0 mirroring. The containers |sdd|infra|4|zfs|/var/lib/lxd/storage-pools/infra|infrastructure (email,dns,usw)|
|sde|infra|5|zfs| | mirror |
-On kb2018 the second pair of disks are Solid State. The first partition on each is a mirrored pair for the infrastructure zfs pool. The remaining partitions are for zfs caching. +On kb2018 the second pair of disks are Solid State. The first partition on each is a mirrored pair for the infrastructure zfs pool. The remaining partitions are for zfs caching.
| | | | | | kb2018 disks|
|---|---|---|---|---|-----------------|
@@ -131,37 +225,37 @@ On kb2018 the second pair of disks are Solid State. The first partition on each |sdb2|devel|2|zfs| | zfs cache (proposed) |
### Hardware raid on the DL380
- +
The raid controller on the Dell allows a mixing of hardware raid and direct hot swappable connections. The HP 420i does only hardware raid or direct connections (HBA) but not both. Since we use the hardware raid the remaining disks need to be configured using the ssacli or the raid controllers bios.
See: DudeWhereAreMyDisks
## Containers
- +
Work previously done by standalone servers is now done though LXD managed containers. [#fn1 (1)]
An up to date list of containers is maintained at https://bitbucket.org/suspectdevicesadmin/ansible/src/master/hosts''
## Ansible
- +
Ansible is used to make most tasks reasonable including.
* creating containers
* updating containers
* updating admin passwords and ssh keys.
* accessing
- +
# Tasks: Accessing Hosts
### bs2020/kb2020 ssh access
-The host machines for the containers can be accessed through the admin lan. Currently this is done through ssh redirection. Eventually it will require a vpn connection. Only ssh key access is allowed and root is not allowed to login. To escalate privileges requires sudo. +The host machines for the containers can be accessed through the admin lan. Currently this is done through ssh redirection. Eventually it will require a vpn connection. Only ssh key access is allowed and root is not allowed to login. To escalate privileges requires sudo.
+
+
+#### Current ssh port mappings to vpn.suspectdevices.com
- -#### Current ssh port mappings to vpn.suspectdevices.com - | port | destination |
-|----|-----------| +|----|-----------|
| 22 | bs2020 ssh via admin lan |
| 222 | bs2020 racadm / serial console via ssh |
| 2222 | knight / vpn |
| 22222 |kb2018 ssh via admin lan |
-| 22223 |kb2018 hpILO / serial console via ssh | +| 22223 |kb2018 hpILO / serial console via ssh |
_note: as of a few updates ago you have to tell apples ssh client to use ssh-dss as below_
@@ -202,12 +296,12 @@ _ if the serial port is still in use do the following _ </>hpiLO-> stop /system1/oemhp_vsp1
### bs2020/kb2018 graphical console access
- +
bs2020 allows complete control of the system via a Dell Idrac 6 controller. This also requires access to the admin lan. This is described on the [wiki:NotesOnIdrac6 Idrac 6 page]
kb2020 allows similar using the on board described on the [wiki:NotesOnILO3 ILO 3 Notes page.]
- +
### ssh access to containers
- +
The susdev profile adds ssh keys and sudo passwords for admin users allowing direct ssh access to the container.
steve:~ don$ ssh feurig@ian.suspectdevices.com
@@ -221,7 +315,7 @@ The containers can be accessed directly from the lxc/lxd host as root ## Updating dns
- +
Dns is provided by bind , The zone files have been consolidated into a single directory under /etc/bind/zones on naomi (dns.suspectdevices.com).
root@naomi:/etc/bind/zones# nano suspectdevices.hosts
@@ -244,7 +338,7 @@ Dns is provided by bind , The zone files have been consolidated into a single di ## Updating Hosts / Containers
- +
When updates are available Apticron sends us an email. We prefer this to autoupdating our hosts as it helps us maintain awareness of what issues are being addressed and does not stop working when there are issues. All hosts in /etc/asnsible/hosts on kb2018 shoul be updated using the following add hoc command.
@@ -261,13 +355,13 @@ https://bitbucket.org/suspectdevicesadmin/ansible/src/master/files/update.sh https://bitbucket.org/suspectdevicesadmin/ansible/src/master/roles/create_lxd_containers/tasks/main.yml
.....YOU ARE HERE.....
_documenting the ansible script to create containers_.
- -## Backing Up Containers - +
+## Backing Up Containers
+
Backing up containers using ansible is depreciated. A python script and cron tab create nightly snapshots and moves them to bs2020.
cd /etc/ansible ;screen -L time ansible-playbook playbooks/backup-lxd-containers.yml -vvv -i importants
https://bitbucket.org/suspectdevicesadmin/ansible/src/master/roles/snapshot_lxd_containers/tasks/main.yml
- +
# links.... (tbd)
|