Openstack

What does it mean

aodh                  Alarming service
barbican              REST API designed for the secure storage, provisioning and management of secrets such as passwords, encryption keys and X.509
cinder                OpenStack Block Storage
gnocchi               Time Series Database as a Service
glance                OpenStack Image Service
heat                  iac. Deploy instances, volumes and other OpenStack services using YAML based templates.
horizon               Openstack’s Dashboard, which provides a web based user interface to OpenStack services
ironic                bootstrap
keystone              identity service 
magnum                container orchestration engines
mistral               workflow service
neutron               networking as a service
nfv                   Network functions virtualization
nova                  cloud computing instance controller, provision compute instances (aka virtual servers).
octavia               Load balancer. Octavia HAProxy Amphora API
rhosp                 Red Hat OpenStack Platform
swift                 OpenStack Object Storage
tacker                NFV Orchestration
trove                 Database as a Service
Zaqar                 multi-tenant cloud messaging service

bash completion

. <(openstack complete 2>/dev/null )

flavor

Which machine types exist.

gp      "General purpose" Well rounded combination of amount of CPUs and the amount of RAM.
hm      "High memory" Optimimzed for applications that need a lot of memory.
hp      "High performance" High frequenzy cpu.

List suitable flavors

openstack flavor list -f json -c VCPUs -c RAM -c Disk -c Name | jq -r '.[] | [ .Name, .VCPUs, .RAM, .Disk ] | @tsv' | sort -k 2,2n -k 3,3n | grep -vE 'gpu|pinned|nvme' | column_tab

Manage multiple projects

export OS_CLIENT_CONFIG_FILE=~/.config/openstack/project1_clouds.yaml
export OS_CLOUD=project1

list volumes

openstack volume list

install openstack

dnf install python3-openstackclient
dnf -y install $(dnf -q search python3- client |grep -i OpenStack | grep -v -- -tests| awk '{print $1}')

get ip addresses of all host

openstack server list -c Networks -f json | jq -r '.[].Networks[][]'

long output

openstack server list --long

select server depending status

openstack server list --status ACTIVE

output

-f csv,json,table,value,yaml

Get output without headers.

openstack server list -f value

list all

openstack command list -f yaml | grep - | grep list | sed 's/^  - /openstack /g' |grep -v "openstack command list" | while read i ; do echo '*' $i ; $i 2>&1 ; done | tee /tmp/openstack_list_resources.${OS_CLOUD}.$(date +%F_%H-%M-%S)
openstack command list -f json | jq -r '.[].Commands[]|select (match("list$"))' | while read i ; do echo '*' openstack $i ; openstack $i 2>&1 ; done | tee /tmp/openstack_list_resources.${OS_CLOUD}.$(date +%F_%H-%M-%S)

How to reach nodes

OPENSTACK=$(openstack server  list -c Networks -f json | jq -r '.[].Networks[][]') ; OPENSTACK_JUMP=$(grep 185 <<< "${OPENSTACK}") ; NODES=$(grep -v 185 <<< "${OPENSTACK}") ; for NODE in ${NODES} ; do echo ssh -J core@${OPENSTACK_JUMP} core@${NODE} ; done

create server

openstack server create --flavor gp.1x2 --availability-zone europe-se-1a --image fedora-37-x86_64 --boot-from-volume 100 --network abjorklund-01-5tsbc-openshift --security-group ssh_allow --key-name abjorklund_ed25519 abjorklund_$(date_file)

With setting password.

cat << EOF > user-data
#cloud-config
password: Password123!
chpasswd: {expire: False}
ssh_pwauth: True
EOF
openstack server create --flavor gp.1x2 --availability-zone europe-se-1a --image rocky-8-x86_64 --boot-from-volume 30 --network abjorklund-01-bmc7w-openshift --security-group ssh_allow --key-name abjorklund_ed25519 abjorklund_$(date_file) --user-data user-data

get router ip

List routers

openstack router list

Get external ip.

openstack router show abjorklund-01-5tsbc-external-router -c external_gateway_info -f json | jq '.external_gateway_info.external_fixed_ips[0].ip_address'
185.102.213.238

Remove subnet from router

openstack router remove subnet <router> <subnet>

list available images

openstack image list

Download image

Get info about image.

openstack image list | grep -i nord-ic-
| 98c03b69-4ba8-4276-8695-b6c3f006cf20 | nord-ic-bc84t-rhcos            | active |
glance image-download --file nord-ic-bc84t-rhcos --progress 98c03b69-4ba8-4276-8695-b6c3f006cf20

Upload image

openstack image create --disk-format qcow2 --container-format bare --public --file CentOS-7-x86_64-GenericCloud-1503.qcow2  CentOS_7_Cloud_IMG

security

Create security group.

openstack security group create <group>

Get security groups.

openstack security group list -c Name -f json | jq -r '.[].Name'

Get all security groups with rules.

openstack security group list -c Name -f value | while read SECURITY ; do openstack security group show "$SECURITY" ; done > /temp/${OS_CLOUD##*/}}_openstack_security_group_list_openstack_security_group_show.$(date_file).log

Get rules from one security group.

openstack security group rule list <group>

Add rule to allow traffic from ip.

openstack security group rule create --proto tcp --dst-port 6443 --remote-ip 185.53.164.10/32 --ingress <group>
openstack security group rule create --proto tcp --dst-port 22 --remote-ip 0.0.0.0/0 --ingress <security group>

Allow nfs4 traffic

# List group to add too.
openstack security group list | grep rw-core
openstack security group rule create --proto tcp --dst-port 2049 --remote-ip 10.2.0.0/16 rw-core-p9dq6-master

Add security group to server

openstack server add security group <server_name_or_id> <security_group_name_or_id>

create block device

openstack volume create --size 50 --type ssd --description "nfs storage block device 0" nfs_storage_abjorklund-01

Resize block device.

os volume set --size 60 nfs_storage_abjorklund-01 --os-volume-api-version 3.42

set physical ip on host

openstack port list

List in different view.

openstack port list -f json | jq -r '.[]|[.ID, .Name, .Status, ."Fixed IP Addresses"[].ip_address]| join("\t")' | column -t -s $'\t'

Change security group on port

Remove if existing port does not exist.

openstack port set --no-security-group a7434863-fc4d-46ad-b93e-b0f2f717023f
openstack port set --security-group 3723f737-280f-453e-af0b-50aca4ce1b0d a7434863-fc4d-46ad-b93e-b0f2f717023f

create port

openstack port create --network abjorklund-01-h4sxm-openshift --fixed-ip subnet=4bb2ab0c-f8f9-4346-b238-5f992f0bcf56,ip-address=10.1.0.5 abjorklund-01-h4sxm-api-port

manage loadbalancer aurora/haproxy

openstack loadbalancer
openstack loadbalancer list
openstack loadbalancer show test-lb -c listeners -f value

view limits

If you have problems to create something in openstack it could be worth verifying you are within limits.

openstack limits show --absolute -f value | grep -E 'RAM|Cores'
openstack quota show --usage -c Resource -c Limit -c "In Use"

count resources

openstack quota show --usage -c Resource -c "In Use" -f json | jq -r '.[] | select(.Resource == "cores" or .Resource == "ram" or .Resource == "gigabytes" )| [.Resource, ."In Use"] | @tsv' | column -t -s $'\t'

get project id

openstack project list -f value -c ID

manage s3/swift

create s3 credentials

openstack ec2 credentials create
openstack ec2 credentials list

Create s3 bucket

aws s3api create-bucket --bucket <bucket>
swift post <bucket>

list s3 storage

swift list blabla/blabla
swift list
aws s3api list-buckets
openstack container list

mount s3 bucket

export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID ; export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY ; export AWS_DEFAULT_REGION=us-west-2 ; s3fs -f -d openshift-region /mnt/openshift-region/ -o endpoint=europe-se-1 -o "host=https://object-eu-se-1a.binero.cloud" -o use_path_request_style

copy file to s3

openstack server show $(openstack server list -f value | head -1 | awk '{print $2}') -c project_id -f value

whoami

openstack configuration show -f json | jq -r '."auth.username"'

sort

Sort on column name.

openstack network list --sort-column Name

selected columns

Select column name only

openstack server list -c Name -c Status -f table

create ssh public key(keypair)

openstack keypair create --public-key /home/abjorklund/.ssh/id_ed25519.pub binero_abjorklund_id_ed25519

List keys

openstack keypair list

Delete kaypair

openstack keypair delete <keypair>

create floating ip

openstack floating ip create europe-se-1-1a-net0

remove floating ip

openstack floating ip delete 193.93.251.72

assign floating ip to port

openstack floating ip set --port abjorklund-01-h4sxm-ingress-port 193.93.251.233

unassign floating ip to server

openstack server remove floating ip ocp-13-nkvgn-master-0 193.93.251.72

assign ip to server

openstack server add floating ip binero_abjorklund_dns-lookup 193.93.248.34

restart server

openstack server reboot <server>
openstack server reboot --hard <server>

view events/log from server

openstack server event list <server>
openstack server event show <server> <requestid>

recovery of server using iso

Upload iso recovery.

openstack image create ubuntu-22.04.4-live-server-amd64.iso --property hw_rescue_device=cdrom --property hw_rescue_bus=scsi --disk-format iso --file ubuntu-22.04.4-live-server-amd64.iso --private --progress
openstack image create Rocky-9.3-x86_64-minimal.iso --property hw_rescue_device=cdrom --property hw_rescue_bus=scsi --disk-format iso --file Rocky-9.3-x86_64-minimal.iso --private --progress

If special properties are not set then set them.

openstack image set --property hw_rescue_device=cdrom --property hw_rescue_bus=scsi <image>

Boot server with recovery iso.

openstack --os-compute-api-version 2.87 server rescue --image Rocky-9.3-x86_64-minimal.iso sentry_rw
openstack --os-compute-api-version 2.87 server rescue --image ubuntu-22.04.3-live-server-amd64.iso sentry_rw

When done disable rescue mode.

openstack server unrescue SERVER

subnet

List subnets

openstack subnet list

hypervisor

Which underlying hypervisor is used.
Not of your business. Ask worthy person.

volume backups

openstack volume backup list

create snapshot

SERVER=<server> openstack server image create --name "backup-${SERVER}-$(date_file)" ${SERVER}

volume snapshot

openstack volume snapshot list
openstack volume snapshot list -c ID -f value | while read SNAPSHOT ; do echo '*' $SNAPSHOT ; openstack volume snapshot delete $SNAPSHOT ; done

dns

# https://docs.binero.com/dns.html
# openstack dns. 
openstack zone create --email anden@halfface.se binero.halfface.se.
# Add entries.
openstack recordset create --record 10.1.0.62  --type A binero.halfface.se. master-0
openstack recordset create --record 10.1.0.249 --type A binero.halfface.se. master-1
openstack recordset create --record 10.1.0.156 --type A binero.halfface.se. master-2
# list enries.
openstack recordset list binero.halfface.se.
# Delete entries.
openstack recordset delete binero.halfface.se. master-2.binero.halfface.se.binero.halfface.se.

Networking explained

1. openstack_networking_network_v2
- Purpose: Represents a network in OpenStack.
- Interaction: This is the top-level network entity. You create a network to provide a layer 2 domain for your instances.

2. openstack_networking_subnet_v2
- Purpose: Represents a subnet within a network.
- Interaction: A subnet is associated with a network. It defines a range of IP addresses, and optionally, DHCP settings.

3. openstack_networking_router_v2
- Purpose: Represents a virtual router.
- Interaction: Routers are used to route traffic between different subnets or networks. They can also be used to provide external network access to instances.

4. openstack_networking_router_interface_v2
- Purpose: Connects a router to a subnet.
- Interaction: This resource is used to add an interface to a router, effectively connecting the router to a subnet, enabling routing between that subnet and others.

5. openstack_networking_port_v2
- Purpose: Represents a port on a network.
- Interaction: Ports are attachment points for devices (like instances) on a network. They have associated IP addresses and MAC addresses.

6. openstack_networking_port_secgroup_associate_v2
- Purpose: Associates security groups with a port.
- Interaction: Security groups define the firewall rules applied to ports. Associating a security group with a port applies the security group's rules to that port.

7. openstack_networking_secgroup_v2
- Purpose: Represents a security group.
- Interaction: Security groups contain a set of rules that define the allowed ingress and egress traffic to and from ports.

8. openstack_networking_secgroup_rule_v2
- Purpose: Represents a rule within a security group.
- Interaction: Security group rules define specific ingress or egress traffic allowed to the instances associated with ports to which the security group is applied.

9. openstack_networking_floatingip_v2
- Purpose: Represents a floating IP.
- Interaction: Floating IPs are IP addresses that can be dynamically associated with instances. They provide external access to instances.

10. openstack_networking_floatingip_associate_v2
- Purpose: Associates a floating IP with a port.
- Interaction: This resource links a floating IP to a port, providing the instance with an external IP address that is accessible from outside the OpenStack cloud.

Interaction Summary
1. Network and Subnet Creation:
  - Create a network using `openstack_networking_network_v2`.
  - Create a subnet within this network using `openstack_networking_subnet_v2`.

2. Router Configuration:
  - Create a router using `openstack_networking_router_v2`.
  - Add an interface to this router to connect it to a subnet using `openstack_networking_router_interface_v2`.

3. Instance Connectivity:
  - Create a port on the network using `openstack_networking_port_v2`.
  - Attach this port to an instance.

4. Security:
  - Create security groups using `openstack_networking_secgroup_v2`.
  - Define rules for the security group using `openstack_networking_secgroup_rule_v2`.
  - Associate security groups with the port using `openstack_networking_port_secgroup_associate_v2`.

5. External Access:
  - Create a floating IP using `openstack_networking_floatingip_v2`.
  - Associate the floating IP with the port attached to the instance using `openstack_networking_floatingip_associate_v2`.

Extend disk

Look at usage from beginning.

df -lh

Extend partition partprobe

openstack volume set --size <new_size> <volume> --os-volume-api-version 3.42

Grow partition that you intend to grow. This part can be ignored if using disk as raw partion(/dev/sd?)

growpart /dev/sdb 1

Resize filesystem.

resize2fs /dev/sdb1

Look at usage after extend.

df -lh

affinity

affinity           Put servers on same hypervisor.
anti-affinity      Dont put servers on same hypervisor.
soft-affinity      Put servers on same hypervisor if possible.
soft-anti-affinity Put server on hypervisor if possible

Are hosts located on same hypervisor

openstack server list -f value -c ID | while read i ; do openstack server show $i -f value -c hostId -c name | xargs ; done | sort -n

Define how server should be handled regarding affinity

openstack server group list

create server group name with soft affinity

openstack server group create --policy soft-anti-affinity <name>

install openstack cli on rhel7/8

yum install centos-release-openstack-train.noarch
yum install python-openstackclient
sed -i 's/^mirrorlist/#mirrorlist/g' /etc/yum.repos.d/*
sed -i 's%^#baseurl=http://mirror.centos.org%baseurl=http://vault.centos.org%g' /etc/yum.repos.d/*

remove network with dependencies

export ROUTER=93b6d9c1-67db-4e04-9aaf-dad208def3c4
export NETWORK=9b86febc-f46f-4990-96c8-ef2e4bc8e139
openstack router remove subnet ${ROUTER} $(openstack route show ${ROUTER} -f json | jq -r '.interfaces_info[].subnet_id')
openstack router unset --external-gateway ${ROUTER}
for port in $(openstack port list --network ${NETWORK} -f value -c ID); do
  openstack port delete "$port"
done
for subnet in $(openstack subnet list --network ${NETWORK} -f value -c ID); do
   openstack subnet delete "$subnet"
done
openstack network delete ${NETWORK}
openstack router delete ${ROUTER}