infrastructure

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
infrastructure [2023-01-14 00:16] – [Dockerize services] asdfinfrastructure [2023-05-08 06:04] (current) – asdf
Line 15: Line 15:
  
 ===== Future ideas ===== ===== Future ideas =====
 +<WRAP center round todo 60%>
 +Migrate this section to [[:rnd:sketchpad]]?
 +</WRAP>
  
 ==== NextCloud instance ==== ==== NextCloud instance ====
Line 20: Line 23:
  
 The current plan is to install the instance on a Raspberry Pi, which will allow me to have full hands-on access to the storage media. This may complicate global access, as I have little control over port forwarding at my current residence. See "5G Hotspots and Tinc" in 38:1 of 2600 for a possible solution to this exact problem. The current plan is to install the instance on a Raspberry Pi, which will allow me to have full hands-on access to the storage media. This may complicate global access, as I have little control over port forwarding at my current residence. See "5G Hotspots and Tinc" in 38:1 of 2600 for a possible solution to this exact problem.
 +
 +Currently installed on the Raspberry Pi and the reverse proxy is functional. <del>Note that some finagling will be necessary to get CalDAV to work; see [[https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#nginx|here]] for NGINX instructions.</del> This expected finagling seems to have been unnecessary, as bidirectional calendar sync is working just fine without it. The next step is to prepare the hard drive that will back the data store, mount it at ''/srv/nextcloud'', and point the containers at it. 
 +
 +I'm not thrilled by the speeds I've seen thus far, but it's hard to tell where the bottleneck is. Ping times through the VPN from my laptop to the Pi average ~350 ms, and with NextCloud configured for reverse proxy, it's impossible to test its local performance (even navigating to its internal IP redirects to sinensis). The problem could simply be CPU usage at the node itself. It's not that big a deal, but it does warrant notice.
 +
 +==== Airflow ====
 +A DAG-based automation tool. Think cron, but with task designations arranged in Python and built as graphs. What I'd like to do is migrate my JQL monitoring script to Airflow and have it [[https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html#task-generated-mapping|dynamically generate]] the notification tasks. The same general outline can be applied to other watch tasks (e.g. watching product sites to tell me when they're available). During my quick experimentation session, I developed a custom Gotify operator for Airflow that I can leverage for future work.
 +
 +The problem is that Airflow seems to be fairly CPU-intensive. The basic docker-compose config easily maxed out my laptop's processor all on its own, so it will absolutely murder sinensis. A Raspberry Pi (with large RAM) might do the trick---I'll just need to tunnel the webserver for roaming access. 
  
 ==== Glances instance ==== ==== Glances instance ====
Line 32: Line 44:
 Unfortunately, this will not be able to run on sinensis, at least not without some hardware upgrades. Maybe I can run this in Docker as well? Unfortunately, this will not be able to run on sinensis, at least not without some hardware upgrades. Maybe I can run this in Docker as well?
  
 +[[tinc]] combined with an [[nginx]] reverse proxy has proven to be successful for making the server public. Adding the following snippet to ''nginx.conf'' will suffice:
 +
 +<code>
 +stream {
 +    server {
 +        listen ADDR;
 +        proxy_pass minecraft;
 +    }
 +    
 +    upstream minecraft {
 +        server ADDR:PORT;
 +    }
 +}
 +</code>
 ==== Mininet ==== ==== Mininet ====
 A small [[Raspberry Pi]] cluster for hosting my experiments into distributed computing. A small [[Raspberry Pi]] cluster for hosting my experiments into distributed computing.
  • infrastructure.1673655368.txt.gz
  • Last modified: 2023-01-14 00:16
  • by asdf