current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 2 down vote favorite

I'd like to rewrite the source IP on TCP/514 traffic leaving a redhat machine, for connections that weren't initiated from the machine.

The machine receives TCP/514 traffic on an interface, for example 10.10.0.20, and then I'd like to return the traffic as though the reply is from 10.10.0.15 (which isn't assigned to the machine).

If I was initiating the connection, then I could use the nat table, and:

iptables -A POSTROUTING -t nat -p tcp --sport 514 -j SNAT --to=10.10.0.15

..but since I'm replying to incoming traffic, I can't make it hit the nat table (as far as I can tell). Ignoring the reasons why I need to do things this way, how can I make this work?

More background:

It's a redhat 7 machine sitting behind a Netscaler VIP which receives syslog traffic over TCP (not UDP). I'm using client IP passthrough on the VIP. Due to the firewall seeing return traffic coming from the syslog server IP, not the VIP's IP, the firewall is dropping the traffic, and hence I'd like to rewrite TCP replies from the syslog server so they come from the VIP's IP address. Since the traffic doesn't originate from the backend server, I don't seem to be able to use the nat table (and therefore no -j SNAT).

What I see now is:

13:13:45.439683 IP 10.10.0.8.31854 > 10.10.0.20.514: Flags [S], seq 544116376, win 8190, options [mss 1460], length 0
13:13:45.439743 IP 10.10.0.20.514 > 10.10.0.8.31854: Flags [S.], seq 4163333198, ack 544116377, win 14600, options [mss 1460], length 0

What I want to see is:

13:13:45.439683 IP 10.10.0.8.31854 > 10.10.0.20.514: Flags [S], seq 544116376, win 8190, options [mss 1460], length 0
13:13:45.439743 IP 10.10.0.15.514 > 10.10.0.8.31854: Flags [S.], seq 4163333198, ack 544116377, win 14600, options [mss 1460], length 0
share|improve this question
    
What topology is your VIP network? In common configuration the VIP box does DNAT to the syslog server, i.e. it rewrites destination address of incoming traffic and source address of return traffic. If you want to perform DSR, the VIP box should forward traffic to the syslog server with packets keeping the VIP address for destination. –  yaegashi Aug 19 at 7:24
    
It's does DNAT, but not SNAT, so traffic reaching the syslog server has the new (correct) destination IP of the syslog server (10.10.0.20 in my example) but still has the source IP of the client, rather than the Netscaler's VIP outbound IP. The problem is that the default gateway on the syslog server isn't the Netscaler, so traffic doesn't go back to the Netscaler - it goes direct to the firewall - it's asymmetrically routed. I realise this isn't ideal, but it's a hosted environment and I'm unable to change the topology. –  Daneel Aug 19 at 11:41
    
I should add that the reason I can't use SNAT on the Netscaler is that the syslog server needs the host IP in order to separate traffic by host. We deal with some syslog traffic which has no host information in it, and the only way to get it is from the source IP of the traffic. –  Daneel Aug 19 at 11:49
    
The most efficient way would be to configure Direct Server Return mode correctly on Netscaler, where Netscaler does MAC based forwarding to the syslog server with destination address unchanged (10.10.0.15). Anyway I post my answer to workaround your DNAT configuration and topology. –  yaegashi Aug 19 at 23:56
    
DSR is an option. I'll experiment with this first, as the VIP's frontend IP sits outside the subnet that the syslog server is in (behind a firewall). –  Daneel Aug 20 at 1:12

1 Answer 1

up vote 2 down vote accepted

DSR method

The most efficient way would be to configure Direct Server Return mode correctly on Netscaler, where Netscaler does MAC based forwarding to the syslog server with destination VIP address unchanged (10.10.0.15).

The syslog server also needs to have that VIP address in order to receive packets forwarded from Netscaler. The address can be assigned to any internal interface like lo or dummy0.

ip addr add 10.10.0.15/32 dev lo

And you have to set some sysctls on the incoming interface (here I assume eth0) to avoid problems with ARP for VIP (see 6.7. The Cure: 2.6.x kernels - arp_ignore/arp_announce). Add the following in /etc/sysctl.conf and run sysctl -p.

net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2

Note that it's useless to set arp_ignore / arp_announce on lo.

DNAT method

If Netscaler does DNAT on the incoming traffic, then the syslog server should definitely forward the return traffic to Netscaler as well so that it could free connection tracking resource. That's the most natural way to complete address translation.

In this case you might want to utilize the policy routing on the syslog server. With this you can apply a special routing table to pakcets in the specific condition like "outgoing TCP packets from port 514".

There are HOWTO docs on Linux advanced routing like this. I recommend you to look through the latter mini HOWTO to understand the following instruction.

First, define the special routing table named VIP with any ID in /etc/iproute2/rt_tables:

1 VIP

Add a default route to VIP (10.10.0.15) to this VIP table:

ip route add default via 10.10.0.15 table VIP

Add an entry to iptables mangle table to mark 1 on outgoing TCP packets from port 514:

iptables -t mangle -A OUTPUT -p tcp --sport 514 -j MARK --set-mark 1

Add a rule to look up VIP routing table on packets with mark 1:

ip rule add from all fwmark 1 table VIP

You can see rules defined so far by ip rule list. Rules are processed in ascending order of priority value (0 is highest precedence).

# ip rule list
0:  from all lookup local 
32765:  from all fwmark 0x1 lookup VIP 
32766:  from all lookup main 
32767:  from all lookup default

You can check the content of each routing table like this:

# ip route ls table local
# ip route ls table VIP
# ip route ls table main
share|improve this answer
    
Thanks for the detailed answer. I'm holding off accepting it for now, as the VIP sits outside the subnet that the syslog server is in (and hence behind a firewall, which would drop the traffic). I don't have enough rep to upvote your answer, but I greatly appreciate the time you took to reply. I'll upvote as soon as I have enough rep. –  Daneel Aug 20 at 1:13
    
Oh, VIP and the syslog server are in different subnets? Then neither this policy routhing trick nor DSR is likely to work. You might well manage to find a way to change the configuration of router/firewall between them... It would be nice if you could configure the policy routhing above on the router. –  yaegashi Aug 20 at 1:44
    
Turns out DSR works a treat. Are you able to make a new answer with DSR as the recommendation so I can accept it? I used support.citrix.com/article/CTX110501 as a reference for how to reconfigure the VIP, and used barracuda.com/support/knowledgebase/50160000000GNC9 for loopback configuration. –  Daneel Aug 20 at 5:09
    
Good! I've updated my answer. Be careful that barracuda's instruction seems incorrect as they are using obsolete or wrong configuration. –  yaegashi Aug 20 at 6:16
    
What I actually wanted to say was that you shouldn't set that sysctl on lo. From LVS-HOWTO: On the realservers the VIP will still be on lo (as for the hidden method). If the reply packets to the client are routed through eth0, then the arp announcements/requests are made through eth0 and you will apply the arp_ignore/arp_announce sysctls to eth0, not to lo (you cannot use arp_ignore/arp_announce on lo). –  yaegashi Aug 20 at 7:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.