MikroTik Web Proxy Setup Guide

Web Proxy is a device that sits between clients and the Internet ( from a network point of view) there are a number of uses for a web proxy server however the following 2 are the most popular uses

• Security
• Performance

Security is provided to the Client because the proxy initiates the connection to the web server on the web clients behalf, thus the client never directly connects to an outside web server. Also with the advent of Proxy Plug-ins such as ICAP Antivirus Appliances, and Content Filtering Software such as Dans Guardian / Web Sense
Performance is accelerated using a web proxy cache, this feature can optimise Internet bandwidth usage by simply caching regularly downloaded files , for example the Google ® Logo and home page. Using a Proxy Cache can improve Internet connection usage by 10% or more depending on human factors and the variety of websites being used by the users of the web cache.
MikroTik say "Web proxy performs Internet object cache function by storing requested Internet objects, i.e., data available via HTTP and FTP protocols on a system positioned closer to the recipient than the site the data is originated from. Here 'closer' means increased path reliability, speed or both. Web browsers can then use the local proxy cache to speed up access and external reduce bandwidth consumption."
MikroTik Router OS 2.9 & 3.0 have a proxy feature that have their origins in Squid Proxy Caching Server. The MikroTik Router OS Proxy server has the following functionality:

• Regular HTTP proxy
• Transparent proxy. Can be transparent and regular at the same time
• Access list by source, destination, URL and requested method (including powerfull regular expression filtering
• Cache access list (specifies which objects to cache, and which notto cache)
• Direct Access List (specifies which resources should be accessed directly, and which resources - through another proxy server)
• Logging facility

MikroTik Proxy Requirements

No Caching: 

• Packages required: web-proxy
• Minimum License required: Level3
• Submenu level: /ip web-proxy
• Standards and Technologies: HTTP/1.0, HTTP/1.1, FTP

Caching:

• All of the above
• Disk
• Memory

 

MikroTik Proxy Recommendation

Toms Tips:

• Caching Internet access will require a lot of read and writes to the disk, chose as fast a speed (RPM) ≥ 7200RPM min disk as possible for maximum performance / concurrent user request support.
• Caching Internet access should only be stored  to RAM or a hard disk, writing to a routers on-board NAND storage or Compact Flash Cards are not recommended as they have a finite number of writes and eventually wear out. If you must use a CF Card try to use an industrial /military grade CF Cards, which deploy ECC(Error Correction Code) and have good "bad sector Management".
• Never use the same storage disk to store your your cache and your your Router OS, to ensure there is always enough space on your router OS Disk for logs, upgrade / update packages & Backups.
• New Solid State Disks offer attractive Performance particularly in relation to random access seek times. However you should use this new technology with due care. Always use high quality enterprise grade disks and install them as a second drive you can easily do this with RouterOS 3x and above using the /system/stores menu option.

Maximum Cache Size:

MikroTik Router OS 2.9.x  

has a memory limited maximum cache size, the calculation of a the cache size can be carried out using the fallowing empirical formula outlined in MikroTik's Router OS 2.9 Web proxy manual

By default the proxy cache can use as much disk space as there is allocated for it. When the system allocates the space for the proxy cache, 1/7th of the total partition (disk) size is reserved for the system, but not less than 50MB. The rest is left for the proxy cache. The system RAM size is considered as well when allocating the cache size. The cache size is limited so, that there are at least 15MB of RAM per 1GB of cache plus 55MB of RAM is reserved for the system. max-cache-size is also taken in account, so the cache will not occupy more than it is specified in this property. The effective limit is calculated as a minimum of all three limits. Note also that RouterOS supports up to 1GB of memory.

Allowing for 64MB of Router OS memory usage one can have 936MB of Memory Available for Cache,

15 MB of ram per 1GB of Cache = (936MB/ 15MB) GB = No of GB Hard Disk required =  62.4GB

The Maximum cache Size of a RouterOS Cache =62.4GB

MikroTik Router OS 3x, 4x, 5x

has a memory limited maximum cache size, the calculation of a the cache size can be carried out using the fallowing empirical formula outlined in MikroTik's Router OS 2.9 Web proxy manual

By default the proxy cache can use as much disk space as there is allocated for it. When the system allocates the space for the proxy cache, 1/7th of the total partition (disk) size is reserved for the system, but not less than 50MB. The rest is left for the proxy cache. The system RAM size is considered as well when allocating the cache size. The cache size is limited so, that there are at least 15MB of RAM per 1GB of cache plus 55MB of RAM is reserved for the system. max-cache-size is also taken in account, so the cache will not occupy more than it is specified in this property. The effective limit is calculated as a minimum of all three limits. Note also that RouterOS supports up to 1GB of memory.

Allowing for 64MB of Router OS memory usage one can have 1936MB of Memory Available for Cache,

15 MB of ram per 1GB of Cache = (1936MB/ 15MB ) GB = No of GB Hard Disk required =  129GB

The Maximum cache Size of a RouterOS V 3.x, 4.x 5.x,  Cache =129GB

Step by Step installation Guide of a Caching Proxy

 

 

Winbox into the Router OS Hardware that you intend to install the webproxy onto, Click on IP \ Web proxy as shown below

• Click on Settings as shown below

• Fill in the following details as shown in the picture below
• Port:-- select 3128 (standard Squid TCP Port) or 8080 (typically used Http proxy server TCP port) however any available port on the Router OS Appliance can be used (provided that the port is not already being used by another process)
• Host name -- Select a host name that you desire (it is not crucial however it is useful for handing out a dns name such as proxy1.wirelessconnect.eu ... (Remember to update your DNS server with the Proxy IP address before issuing the name to clients
• Transparent Proxy -- Tick this Box if the Proxy Server is to be Transparent, ie the user will not be required to configure their browser (note additional firewall configuration (redirect rule  will need to be inserted to make this work see bottom of article for more details)
• Cache Administrator-- Select an Administrative Email-address for receiving feedback on your Proxy Appliance Performance
• Maximum Object Size -- Select a reasonable size (It should be large enough for most users uses ...e.g Service Pack 2 Download ... Patch CD ISO,) however it should not exceed the Size of the Caching Disk (We Recommend that the Maximum Object Cache be a tiny fraction of the total cache size i.e. Maximum Object Size should << 1% of Caching Disk)  

• Select the correct Drive (secondary-master) as the Cache Drive and then click Format as shown below (Note that Router OS wont Let you format the System Drive)
•  

 

When prompted to confirm the formatting  as shown below

 

•  

 

• While the cache drive is formatting , "formatting harddrive"  will appear on the status bar on the bottom of the dialogue box as shown below

• After the formatting process is complete the Cache will be created & "Creating Cache" will appear on the status bar at the bottom of the dialogue box as shown below.

• Select the Maximum RAM Cache Size, this should be no greater than the result of the following formula -- (Total RAM on Proxy Appliance) - 64 MB Ram (For Router OS and Other Router OS Process) in this  example one has an appliance with 1GB of Memory installed and one  wishes to reserve 68 MB of RAM for system use therefore one should set the Maximum RAM Cache Size to 934MB as shown in the image below

• Next Turn on the Proxy Server by clicking Enable as shown below

 

• Once the Proxy Service is running the status bar will show "Running" on the bottom of the dialogue box as shown below

Proxy Running State will be displayed in the Status Bar

Proxy Setup is now complete and now you can instruct your users to configure the proxy server in their web Browsers (or indeed in an enterprise one can set the proxy server using group policy! It is possible to enforce the proxy server transparently (so that you dont have to configure end user systems) this is discussed in the Creating a Transparent Proxy Article