WebAgent 4.1 Not Reverse Proxying Sometimes

[FireBurn]

Going to https://proxysite.example.com/login you’re redirected to https://ssosite.example.com:443/opensso/UI/Login?realm=example&goto=https%3A%2F%2Fproxysite.example.com%3A443%2Flogin

Login appears to work but the https://proxysite.example.com/pages/dashboard/ page doesn’t load, eventually an timeout error appears with “ERR_CONTENT_LENGTH_MISMATCH” (this just shows a blank page with Chrome 61 and other browsers)

In the logs there are messages such as:

[Wed Sep 06 10:49:54.675916 2017] [proxy_http:error] [pid 32553:tid 140342846269184] (70007)The timeout specified has expired: [client 1.2.3.4:57475] AH01110: error reading response
[Wed Sep 06 10:54:17.973986 2017] [proxy_http:error] [pid 32550:tid 140342846269184] (70007)The timeout specified has expired: [client 1.2.3.4:64700] AH01110: error reading response, referer: https://ssosite.example.com:443/opensso/UI/Login?realm=bskyb&goto=https%3A%2F%2Fproxysite.example.com%3A443%2Flogin

Doing a stop and a start of apache seems to fix the issue

I don’t seem to be the only person seeing this issue https://serverfault.com/questions/870185/apache-restart-cause-proxy-httperror-pid-120502-70008partial-results-are/872279#872279

I’ve also raised an issue on the apache bugtracker https://bz.apache.org/bugzilla/show_bug.cgi?id=61501

I’m wondering if there’s a race issue between the OpenAM module and the proxy module

[Peter Major]

@spareyc Might be able to help you here.

[FireBurn]

So I’ve tracked down why it’s not working, something is downgrading GETs to HEADs:

Not working

2017-09-08 15:38:13.595 +0100 DEBUG [0x7f2502e46700:29476][source/apache/agent.c:579] get_method_num(): method GET (HEAD, 0)
2017-09-08 15:38:13.595 +0100 DEBUG [0x7f2502e46700:29476][source/apache/agent.c:588] get_method_num(): number corresponds to HEAD method
2017-09-08 15:38:13.595 +0100 DEBUG [0x7f2502e46700:29476][source/apache/agent.c:607] get_method_num(): set method to HEAD
2017-09-08 15:38:13.595 +0100 DEBUG [0x7f2502e46700:29476][source/apache/agent.c:609] get_method_num(): This is method_num before return: 3

Working:

2017-09-08 15:50:16.939 +0100 DEBUG [0x7f24b37fe700:30733][source/apache/agent.c:579] get_method_num(): method GET (GET, 0)
2017-09-08 15:50:16.939 +0100 DEBUG [0x7f24b37fe700:30733][source/apache/agent.c:588] get_method_num(): number corresponds to GET method
2017-09-08 15:50:16.939 +0100 DEBUG [0x7f24b37fe700:30733][source/apache/agent.c:609] get_method_num(): This is method_num before return: 1

Just need to figure out why

[Peter Major]

Is this it? https://bugster.forgerock.org/jira/browse/AMAGENTS-349

[FireBurn]

This is a very hacky work around:

commit b113a1ab496ea13211d9a937f6f7d792ecd7c4c8 (HEAD -> master, origin/master, origin/HEAD)
Author: Mike Lothian <mike@fireburn.co.uk>
Date: Fri Sep 8 15:10:58 2017 +0100

Extra debugging and force to GET

diff –git a/source/apache/agent.c b/source/apache/agent.c
index 6d150da..b6b723c 100644
— a/source/apache/agent.c
+++ b/source/apache/agent.c
@@ -603,6 +603,13 @@ static char get_method_num(request_rec *r, unsigned long instance_id) {
* the method string needs to be preserved in case Apache is
* used as a proxy (in front of Exchange Server for instance)
*/
+
+ AM_LOG_DEBUG(instance_id, “%s This is method_num before return: %d”, thisfunc, method_num);
+ if (method_num == 3) {
+ method_num = 1;
+ }
+ AM_LOG_DEBUG(instance_id, “%s This is method_num we now return: %d”, thisfunc, method_num);
+
r->method = am_method_num_to_str(method_num);
AM_LOG_DEBUG(instance_id, “%s set method to %s”, thisfunc, LOGEMPTY(r->method));
}

[FireBurn]

Yeah that’s the same issue

[FireBurn]

I think this is the commit in apache that breaks things: https://github.com/apache/httpd/commit/1485d64698ef816f1586e2d772c58cb9b78c788d

[Author]

Posts