User Tools

Site Tools


tamiwiki:internal:networks:tami_sre

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
tamiwiki:internal:networks:tami_sre [2023/03/15 22:35] 444btamiwiki:internal:networks:tami_sre [2023/05/26 21:51] (current) – removed corshunov
Line 1: Line 1:
-====== TAMI Site Reliability Engineering ====== 
-This page details our efforts to keep the systems online reliable 
----- 
  
-Currently, we have a Raspberry pi in the space that runs a realtime [[https://status.telavivmakers.space|Status]] webpage from UptimeRobot 
- 
- 
- 
- 
-===== Troubleshooting steps ===== 
-The first step is to identify the nature of the root cause and whether it is related to the network or the infrastructure. 
-Use the [https://stats.uptimerobot.com/Jx4JQiBDEZ|UptimeRobot site]] to see what could be up or down. 
-  * If everything is down(including Tamis IP, 82.80.54.64), is it definitely a Network issue but maybe also infra 
-  * If the IP address of Tami is reachable (Ping and Telnet), but the yunohost services are down, its likely just an infra issue 
-  * In case the stuff is still not functioning after trying all these steps, you should reach out to someone from the [[tamiwiki/external/contact | contact]] page. Please note that all the email, matrix and XMPP is hosted inside tami so you cant comm on those channels. Reach out via telegram and post in main channel 
- 
-==== Network ==== 
-Relevant Link: [[tamiwiki:internal:networks:digital_infra|Physical Infra]] 
- 
-==== Infra ==== 
-Relevant Link: [[tamiwiki:internal:networks:physical_infra|Physical Infra]] 
-=== If there is an issue with a single service === 
-  * The first step is to see if you can log into [[https://telavivmakers.space/yunohost/admin/#/|yunohost admin panel]] 
-  * Then check the service at [[https://telavivmakers.space/yunohost/admin/#/tools|Tools]] > [[https://telavivmakers.space/yunohost/admin/#/services|Services]] 
-  * Review the logs, restart the service if necessary and maybe share logs with yunopast into a relevant group in tamis communication channel 
-=== If there is an issue with a multiple services === 
-  * Attempt the steps above for each services but if its all services, it might be something related to yunohost or the device it is running on 
-  * Try to ssh into yunohost. The password is your yunohost SSO password 
-    * ssh <yunohost username>, telavivmakers.space 
-  * Check out the output from the following services 
-    * sudo systemctl status nginx.service (for website issues) 
-    * sudo systemctl status mautrix_telegram.service (for telegram bridge issues) 
-  * For any errors or for any other reason, try restart the service if it is already broken) 
-    * sudo systemctl restart <relevant service> 
-  * Failing this, try look for more logs. Look up any error messages and go down the rabbit holes 
-    * sudo journalctl -u <relevant service> 
- 
- 
- 
- 
----- 
-QR Code for page, do not delete 
- 
-{{:tamiwiki:internal:networks:tami_sre.png?nolink&200 |}} 
tamiwiki/internal/networks/tami_sre.1678912507.txt.gz · Last modified: 2023/03/15 22:35 by 444b