I have a proxy server, it will have a captive portal page. Users will be required to sign in before gaining access to the internet.
So far I am aware of IP and/or MAC address based authentication.
Is it possible to use HTTP cookies to implement authentication? (Maybe third party cookies?) How does websense do it? (see cookie authentication)
If HTTP cookies are not usable for proxy authentication, is MAC address authentication the industry standard for this thing?
Typically, the standard way to implement proxy authentication is, surprisingly, Proxy Authentication.
This is a set of matching response and request headers, that together implement authentication that is required between the user and the proxy.
This is commonly used with transparent proxies, and is well supported in both browsers and proxies. Some proxies do implement this differently (e.g. how the credentials are verified), but in most cases this is the much preferred solution. There are very few drawbacks to this solution, and alternate solutions that break standard HTTP, such as BlueCoat, have quite a few issues.
Now, you do mention that your proxy will be implementing a captive portal, but you did not mention why. If it is just to implement the authentication form, thinking that there is no built-in solution, I would recommend you change your solution, use standard proxies, and just define required authentication. Building your own captive portal is just way too tricky, and will likely open you up to a whole slew of vulnerabilities (from experience).
If there is some other reason to build the captive portal - random signups, payment options, crazy fun, etc - perhaps elaborate, and we can deal with them. Otherwise it does look like an XY problem (where you assume a solution and ask about that, instead of asking about your actual problem...)
BlueCoat when deployed as a transparent proxy can use the concept of cookie surrogates to authenticate a user, in order to make it work the core mechanism used is the redirection to a virtual URL that will always request authentication (HTTP 401), here's how it's implemented on it:
First time authentication:
1. UA (User agent) issues a GET / to HOST:www.example.com
2. Proxy server intercepts the GET and replies with a redirect (302) to the virtual URL and a query string that contains the original URL requested, e.g. "http://virtualURL.com?QueryString"
3. UA attempts to connect to virtual URL
4. Proxy replies with HTTP 401, requesting authentication
5. UA reissues the same request of step 3, but with authentication credentials
6. UA is redirected again (302), this time to the original URL, the query string is still attached to this redirect (http://www.example.com?QueryString), also there's a SET-COOKIE that appears to be coming from virtualURL (this will be used on requests to sites other than the initial one)
7. UA connects to the original URL + Query string, since it still have the Query String on the request the proxy issues a new 302, this time to www.example.com, also includes a SET-COOKIE, but this time the cookie behaves as being issued from example.com domain.
8. UA connects to www.example.com, the cookie is also received and accepted. This cookie notifies the proxy that the user is authenticated.
9. The proxy forwards the UA request to the original URL removing the cookie, to not interfere with the transaction.
Subsequent requests will follow as:
Request to www.newsite.com, get redirected to virtualURL?QueryString, this time there already is a cookie to the virtualURL domain, so no 401 is needed, new redirect to newsite.com?QueryString, SET-COOKIE for new domain, redirect to original URL.
I guess that using the same concept you can adapt it to your proxy server.
