A New Method for Variable Elimination in Systems of Inequations

Farhad Shirani Chaharsooghi, Mohammad Javad Emadi, Mahdi Zamanighomi and Mohammad Reza Aref Information Systems and Security Lab (ISSL)
Electrical Engineering Department, Sharif University of Technology, Tehran, Iran.
Email: fshirani@ee.sharif.edu, emadi@ee.sharif.edu, m_zamani@ee.sharif.edu, aref@sharif.edu
Abstract

In this paper, we present a new method for variable elimination in systems of inequations which is much faster than the Fourier-Motzkin Elimination (FME) method. In our method, a linear Diophantine problem is introduced which is dual to our original problem. The new Diophantine system is then solved, and the final result is calculated by finding the dual inequations system. Our new method uses the algorithm Normaliz to find the Hilbert basis of the solution space of the given Diophantine problem. We introduce a problem in the interference channel with multiple nodes and solve it with our new method. Next, we generalize our method to all problems involving FME and in the end we compare our method with the previous method. We show that our method has many advantages in comparison to the previous method. It does not produce many of the redundant answers of the FME method. It also solves the whole problem in one step whereas the previous method uses a step by step approach in eliminating each auxiliary variable.

I Introduction

The FME Method was first introduced by Fourier [1] in 1827 and was rediscovered in Motzkin’s thesis in 1936 [2]. Considering the set of inequations Axb,ARm,n,bRnformulae-sequence𝐴𝑥𝑏formulae-sequence𝐴superscript𝑅𝑚��𝑏superscript𝑅𝑛Ax\leq b,A\in R^{m,n},b\in R^{n} the method provides a scheme for determining the existence of a real or integer solution to the given problem. The method uses a step by step algorithm, in which each step reduces the dimension of the answer space by projecting the answer space on a hyperplane. The procedure continues until the dimension is reduced to one; if the one dimensional answer is feasible, then the solution exists. A dual mode for FME was introduced in 1972 by Dantzik [3].

The FME method is widely used in solving rate region problems since there is often the need to eliminate the auxiliary variables introduced into the problem from the resulting inequations and hence reduce the dimension of the solution space by finding its projection on a given plane. The complexity of calculation in each step is of the order of k24superscript𝑘24\frac{k^{2}}{4}, with k𝑘k being the number of constraints in the system of inequalities. This complexity makes the method unfeasible for high dimensional problems. There have been many attempts to decrease the calculation time of the method. In 1996, Kebler [4] introduced a parallel computation method which was more efficient when solving the elimination problem on a computer.

In this paper, we present a new method for eliminating auxiliary variables in inequations which is much faster than the FME method. In our method, we first introduce a linear Diophantine problem which is dual to the original problem. We then proceed by solving this new Diophantine system, next we calculate the final result by finding the dual inequations system. Our new method uses the algorithm Normaliz to find the Hilbert basis of the solution space of the given Diophantine problem.

Normaliz was first introduced as a program in 1997 by R. Koch. In 2003, the algorithm was presented in Koch’s thesis [5]. Normaliz was originally introduced to compute the lattice points in a lattice polytope and the integral closure of a monomial ideal. Koch mentions in his thesis that the algorithm may be extended to find the Hilbert basis of the solution space of a system of linear Diophantine equations. However, applying the algorithm introduced in [5] to information theoritic problems is still time-consuming. In 2010, Bruns and Ichim [6] introduced a dual algorithm which is much faster when applied to problems with a greater number of constraints than auxiliary variables. Since this is the case in almost all information theory problems, we use this method to find the desired Hilbert Basis solution.

One of the most important fundamental channels in communication systems is the Interference Channel (IC). The IC was initiated by Shannon [7]. Simple outer bounds for the IC were established by Ahlswede [8]. Later, Carlieal [9], using superposition encoding and sequential decoding, characterized an achievable rate region for general discrete memoryless IC. In the IC each receiver is only interested in the message from its corresponding sender. However, it might decode messages from other senders to reduce the interference effect. Han and Kobayashi (HK) utilized rate splitting techniques in [10], and applied simultaneous superposition coding techniques to establish the largest inner bound introduced for the general IC to date. Computing the capacity region of general IC is still an open problem, but in special cases such as strong interference regime the capacity region has been established, and the region was the same as the HK rate region [11]. In the method used to achieve the HK rate region, the messages are split into two parts, the private and common parts; private messages are decoded by the corresponding receiver, while the common messages are decoded by all receivers.

Rate splitting has become a prominent tool for solving rate region problems. However, there is a drawback to the method in that solving the channel with the new split messages yields a system of inequations based on the split-rates. Since we are interested in each sender’s total rates, the auxiliary variables must be eliminated. The FME method is needed to eliminate these auxiliary rates.

The rest of the paper is organized as follows: In Section II, we first introduce our model of an l𝑙l transmitter-receiver interference channel. In Section III, we introduce a problem in the IC. In this section, we use some simplifying assumptions, since the goal of the section is to explain the new elimination method. In Section IV, we solve the problem with our new method. In Section V, we present a generalized algorithm for our new method which could be applied to any problem involving FME. In Section VI, we present results of applying our new method to several problems which were solved previously using the FME method. Section VII concludes the paper.

II Channel Moddel

In this section, we introduce our model of IC. We denote random variables by upper case letters e.g. X, their realizations are presented by lower case letters e.g. x in finite sets denoted by 𝒳𝒳\mathpzc{X}. We use xiksubscriptsuperscript𝑥𝑘𝑖x^{k}_{i} to denote the vector xi,xi+1,,xksubscript𝑥𝑖subscript𝑥𝑖1subscript𝑥𝑘x_{i},x_{i+1},...,x_{k}. We denote xksuperscript𝑥𝑘x^{k} if i=1, while for i=1 and k=l𝑙l, we show the vector by x¯¯𝑥\underline{x}. Also, if i=1 and k=2l2𝑙2l, we show the vector by x. Furthermore x¯insuperscriptsubscript¯𝑥𝑖𝑛\underline{x}_{i}^{n} is the compact form of (x¯i,x¯i+1,,x¯n)subscript¯𝑥𝑖subscript¯𝑥𝑖1subscript¯𝑥𝑛(\underline{x}_{i},\underline{x}_{i+1},...,\underline{x}_{n}) where x¯i=(xi1,,xil)subscript¯𝑥𝑖subscript𝑥𝑖1subscript𝑥𝑖𝑙\underline{x}_{i}=(x_{i1},...,x_{il}).

A discrete Interference Channel, with l𝑙\mathit{l} senders and l𝑙\mathit{l} receivers is the 2l𝑙\mathit{l}+1 tuple (𝒳¯,p(y¯|x¯),𝒴¯)¯𝒳𝑝conditional¯𝑦¯𝑥¯𝒴(\underline{\mathpzc{X}},p(\underline{y}|\underline{x}),\underline{\mathpzc{Y}}) where 𝒳𝒾subscript𝒳𝒾\mathpzc{X}_{i} are the input alphabets, 𝒴𝒾subscript𝒴𝒾\mathpzc{Y}_{i} are the output alphabets for i=1,2,…,l𝑙\mathit{l} and p(y¯|x¯)𝑝conditional¯𝑦¯𝑥p(\underline{y}|\underline{x}) is the conditional probability of the channel. Each decoder is only interested in decoding its corresponding sender’s message; however, in order to reduce the interference effect, receivers might decode the message from other senders as well. The channel is memoryless and time invariant in the sense that

p(y¯n|x¯n,y¯n1)=pY¯|X¯(y¯n|x¯n).𝑝conditionalsubscript¯𝑦𝑛superscript¯𝑥𝑛superscript¯𝑦𝑛1subscript𝑝conditional¯𝑌¯𝑋conditionalsubscript¯𝑦𝑛subscript¯𝑥𝑛p(\underline{y}_{n}|\underline{x}^{n},\underline{y}^{n-1})=p_{\underline{Y}|\underline{X}}(\underline{y}_{n}|\underline{x}_{n}).

A ((2nR1,2nR2,,2nRl),n)superscript2𝑛subscript𝑅1superscript2𝑛subscript𝑅2superscript2𝑛subscript𝑅𝑙𝑛((2^{nR_{1}},2^{nR_{2}},...,2^{nR_{\mathit{l}}}),n) code consists of (i) l𝑙\mathit{l} message sets 𝒲𝒾=1,2,,2𝓃𝒾,𝒾[1:l]\mathpzc{W}_{i}={1,2,...,2^{nR_{i}}},i\in[1:\mathit{l}], (ii) l𝑙\mathit{l} encoding functions fu=𝒲𝓊𝒳𝓊𝓃,𝓊[1:l]f_{u}=\mathpzc{W}_{u}\rightarrow\mathpzc{X}^{n}_{u},u\in[1:\mathit{l}], (iii)l𝑙\mathit{l} decoding functions gu(.)g_{u}(.) where W^u=gu(Yun),u[1:l]\hat{W}_{u}=g_{u}(Y_{u}^{n}),u\in[1:\mathit{l}]. We define the average probability of error for this code as Pe(n)=Pr{i=1l(Wi^Wi)}superscriptsubscript𝑃𝑒𝑛𝑃𝑟superscriptsubscript𝑖1𝑙^subscript𝑊𝑖subscript𝑊𝑖P_{e}^{(n)}=Pr\{\bigcup_{i=1}^{\mathit{l}}(\hat{W_{i}}\neq W_{i})\}. A nonnegative rate l𝑙\mathit{l}-tuple (R1,R2,,Rl)subscript𝑅1subscript𝑅2subscript𝑅𝑙(R_{1},R_{2},...,R_{\mathit{l}}) is said to be achievable if there exists a code ((2nR1,2nR2,,2nRl),n)superscript2𝑛subscript𝑅1superscript2𝑛subscript𝑅2superscript2𝑛subscript𝑅𝑙𝑛((2^{nR_{1}},2^{nR_{2}},...,2^{nR_{\mathit{l}}}),n) for which Pe(n)0superscriptsubscript𝑃𝑒𝑛0P_{e}^{(n)}\rightarrow 0 as n𝑛n tends to infinity.

III Problem Statement

In this section, we present our coding strategy which is a generalization of the method used in [10]. We describe the problem in solving the resulting inequations with the conventional FME method. After stating the problem, we introduce our method and compare the results with the former procedure.

Since our goal in this paper is to introduce an alternative elimination method for the FME method, we use some simplifying assumptions in the channel. We consider the channel to be symmetric in the sense that the i𝑖ith sender sends the same amount of information to all receivers other than its own corresponding receiver. This allows each sender to send the same common message to all receivers. We split each message Wisubscript𝑊𝑖W_{i} into common and private parts (Wic,Wip)subscript𝑊𝑖𝑐subscript𝑊𝑖𝑝(W_{ic},W_{ip}). The common part of the message is decoded by all receivers while the private part is decoded only by the i𝑖ith receiver. Another suitable assumption would have been to assume a sense of degradedness between the receivers in the following way: assume that receiver i1subscript𝑖1i_{1} receives all the common massages sent by sender i𝑖i; receiver i2subscript𝑖2i_{2} receives all the common messages received by the receivers other than i1subscript𝑖1i_{1}, and so on. In this case we had to split the message in each sender to l𝑙\mathit{l} common parts and one private part in the following way: split the message Wisubscript𝑊𝑖W_{i} into (Wi(1c),Wi(2c),,Wi(lc),Wip(W_{i(1c)},W_{i(2c)},...,W_{i(\mathit{l}c)},W_{ip}). Wi(lc)subscript𝑊𝑖𝑙𝑐W_{i(\mathit{l}c)} is decoded by the receiver with the least information about the massage from sender i𝑖i; the reciever with the second least information about this message decodes Wi(lc),Wi((l1)c)subscript𝑊𝑖𝑙𝑐subscript𝑊𝑖𝑙1𝑐W_{i(\mathit{l}c)},W_{i(\mathit{(l-1)}c)}, and so on. The rest of the solution would stay the same. Clearly, the symmetric case mentioned above is a special case of this latter case. Since the two cases have the same solution, and for the sake of simplicity, we solve the channel for the symmetric case in this paper.
Codebook Generation:

Let Z1=(U1,U2,,Ul,X1,X2,,Xl,Y1,Y2,,Yl,Q)subscript𝑍1subscript𝑈1subscript𝑈2subscript𝑈𝑙subscript𝑋1subscript𝑋2subscript𝑋𝑙subscript𝑌1subscript𝑌2subscript𝑌𝑙𝑄Z_{1}=(U_{1},U_{2},...,U_{\mathit{l}},X_{1},X_{2},...,X_{\mathit{l}},Y_{1},Y_{2},...,Y_{\mathit{l}},Q), where Q is the time-sharing variable. Let 𝒫1subscript𝒫1\mathpzc{P}_{1} denote the set of all jont p.m.fs p1subscript𝑝1p_{1} on Z1subscript𝑍1Z_{1}, which can be written in the form

p(z1)=p(q)i=1l(p(ui|q)p(xi|ui,q))p(y1,y2,,yl|x1,x2,,xl)𝑝subscript𝑧1𝑝𝑞superscriptsubscriptproduct𝑖1𝑙𝑝conditionalsubscript𝑢����𝑞𝑝conditionalsubscript𝑥𝑖subscript𝑢𝑖𝑞𝑝subscript𝑦1subscript𝑦2conditionalsubscript𝑦𝑙subscript𝑥1subscript𝑥2subscript𝑥𝑙p(z_{1})=p(q)\prod_{i=1}^{l}(p(u_{i}|q)p(x_{i}|u_{i},q))p(y_{1},y_{2},...,y_{l}|x_{1},x_{2},...,x_{\mathit{l}})

For any p1subscript𝑝1p_{1} defined above, generate 2nRic,i[1:l]2^{nR_{ic}},i\in[1:l], i.i.d uicn(wic)subscriptsuperscript𝑢𝑛𝑖𝑐subscript𝑤𝑖𝑐u^{n}_{ic}(w_{ic}) where wic[1:2nRic]w_{ic}\in[1:2^{nR_{ic}}]. For each uicn(wic)subscriptsuperscript𝑢𝑛𝑖𝑐subscript𝑤𝑖𝑐u^{n}_{ic}(w_{ic}), generate 2nRipsuperscript2𝑛subscript𝑅𝑖𝑝2^{nR_{ip}}, i.i.d xin(wip,wic)subscriptsuperscript𝑥𝑛𝑖subscript𝑤𝑖𝑝subscript𝑤𝑖𝑐x^{n}_{i}(w_{ip},w_{ic}). The time-sharing variable has no impact on the rest of the proof and for the sake of brevity, we ignore it until the end of the proof where we bound its cardinality.
Encoding:

At the beginning of each block, the i𝑖ith sender picks related (uicn,xin)subscriptsuperscript𝑢𝑛𝑖𝑐subscriptsuperscript𝑥𝑛𝑖(u^{n}_{ic},x^{n}_{i}) according to j=1np(uic,j)p(xi,j|uic,j)subscriptsuperscriptproduct𝑛𝑗1𝑝subscript𝑢𝑖𝑐𝑗𝑝conditionalsubscript𝑥𝑖𝑗subscript𝑢𝑖𝑐𝑗\prod^{n}_{j=1}p(u_{ic,j})p(x_{i,j}|u_{ic,j}) and sends it.
Decoding:

At the end of each block, the i𝑖ith receiver finds (xin,u1cn,u2cn,,ulcn)subscriptsuperscript𝑥𝑛𝑖subscriptsuperscript𝑢𝑛1𝑐subscriptsuperscript𝑢𝑛2𝑐subscriptsuperscript𝑢𝑛𝑙𝑐(x^{n}_{i},u^{n}_{1c},u^{n}_{2c},...,u^{n}_{lc}) such that:

(xin(wip,wic),u1cn(w1c),u2cn(w2c),,ulcn(wlc))Aϵsubscriptsuperscript𝑥𝑛𝑖subscript𝑤𝑖𝑝subscript𝑤𝑖𝑐subscriptsuperscript𝑢𝑛1𝑐subscript𝑤1𝑐subscriptsuperscript𝑢𝑛2𝑐subscript𝑤2𝑐subscriptsuperscript𝑢𝑛𝑙𝑐subscript𝑤𝑙𝑐subscript𝐴italic-ϵ(x^{n}_{i}(w_{ip},w_{ic}),u^{n}_{1c}(w_{1c}),u^{n}_{2c}(w_{2c}),...,u^{n}_{lc}(w_{lc}))\in A_{\epsilon}

Without loss of generality we consider the case where all senders have sent index 1 as their messages. We investigate all possible scenarios in the first decoder. The decoder must decode messages (w1p,w1c,w2c,,wlc)subscript𝑤1𝑝subscript𝑤1𝑐subscript𝑤2𝑐subscript𝑤𝑙𝑐(w_{1p},w_{1c},w_{2c},...,w_{lc}), the decoding is errorless if w^=(w^1p,w^1c)=(1,1)^𝑤subscript^𝑤1𝑝subscript^𝑤1𝑐11\hat{w}=(\hat{w}_{1p},\hat{w}_{1c})=(1,1). Using the same method as in [12] and by the AEP rule, it is clear that the probability of error approaches zero as n𝑛n\rightarrow\infty (e.g. rate l𝑙l-tuple is achievable) iff the rates satisfy the following set of equations:

R1p+α1R1c+α2R2c++αlRlcI1,α1,α2,,αlsubscript𝑅1𝑝subscript𝛼1subscript𝑅1𝑐subscript𝛼2subscript𝑅2𝑐subscript𝛼𝑙subscript𝑅𝑙𝑐subscript𝐼1subscript𝛼1subscript𝛼2subscript𝛼𝑙R_{1p}+\alpha_{1}R_{1c}+\alpha_{2}R_{2c}+...+\alpha_{l}R_{lc}\leq I_{1,\alpha_{1},\alpha_{2},...,\alpha_{l}} (1)

where

αi{0,1}subscript𝛼𝑖01\alpha_{i}\in\{0,1\}
I1,α1,α2,,αl=I(X1,US;Y1|USc)subscript𝐼1subscript𝛼1subscript𝛼2subscript𝛼𝑙𝐼subscript𝑋1subscript𝑈𝑆conditionalsubscript𝑌1subscript𝑈superscript𝑆𝑐I_{1,\alpha_{1},\alpha_{2},...,\alpha_{l}}=I(X_{1},U_{S};Y_{1}|U_{S^{c}})

and UiUSsubscript𝑈𝑖subscript𝑈𝑆U_{i}\in U_{S} if αi=1subscript𝛼𝑖1\alpha_{i}=1; also, for αi=0subscript𝛼𝑖0\alpha_{i}=0, we have UiUScsubscript𝑈𝑖subscript𝑈superscript𝑆𝑐U_{i}\in U_{S^{c}}. We note that R1R1c=R1psubscript𝑅1subscript𝑅1𝑐subscript𝑅1𝑝R_{1}-R_{1c}=R_{1p}, substituting into (1) we have

R1α1R1c+α2R2c++αlRlcI1,α1,α2,,αl,αi{0,1}formulae-sequencesubscript𝑅1subscript𝛼1subscript𝑅1𝑐subscript𝛼2subscript𝑅2𝑐subscript𝛼𝑙subscript𝑅𝑙𝑐subscript𝐼1subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝛼𝑖01R_{1}-\alpha_{1}R_{1c}+\alpha_{2}R_{2c}+...+\alpha_{l}R_{lc}\leq I_{1,\alpha_{1},\alpha_{2},...,\alpha_{l}},\alpha_{i}\in\{0,1\}

These are the inequations for decoder 1. We rewrite the inequations for all decoders in matrix form:

(100010000010000100010000010000010001|A1____A2________Al)(R1R2RlR1cR2cRlc)(I1,000I1,001I1,111I2,000I2,001I2,111Il,000Il,111)conditionalmatrix100010000010000100010000010000010001matrixmissing-subexpressionmissing-subexpressionmissing-subexpressionsubscript𝐴1missing-subexpressionmissing-subexpression____missing-subexpressionmissing-subexpressionmissing-subexpressionsubscript𝐴2missing-subexpression________missing-subexpressionmissing-subexpressionmissing-subexpressionsubscript𝐴𝑙matrixsubscript𝑅1subscript𝑅2missing-subexpressionmissing-subexpressionsubscript𝑅𝑙subscript𝑅1𝑐subscript𝑅2𝑐missing-subexpressionmissing-subexpressionsubscript𝑅𝑙𝑐matrixsubscript𝐼1000subscript𝐼1001subscript𝐼1111subscript𝐼2000subscript𝐼2001subscript𝐼2111subscript𝐼𝑙000missing-subexpressionmissing-subexpressionsubscript𝐼𝑙111\left(\begin{matrix}1&0&\ldots&0&0\\ 1&0&\ldots&0&0\\ \vdots&\vdots&\ddots&0&0\\ 1&0&\ldots&0&0\\ 0&1&\ldots&0&0\\ 0&1&\ldots&0&0\\ \vdots&\vdots&\ddots&0&0\\ 0&1&\ldots&0&0\\ \vdots&\vdots&\ddots\\ \vdots&\vdots&\ddots\\ 0&0&\ldots&0&1\\ 0&0&\ldots&0&1\\ \end{matrix}\right|\left.\begin{matrix}\\ &&A_{1}\\ \\ \\ \_&\_&\_&\_\\ \\ &&A_{2}\\ \\ \_&\_&\_&\_\\ \vdots&\vdots&\vdots&\ddots\\ \_&\_&\_&\_\\ \\ &&A_{l}\\ \\ \end{matrix}\right)\left(\begin{matrix}R_{1}\\ R_{2}\\ \\ \vdots\\ \\ \vdots\\ R_{l}\\ R_{1c}\\ R_{2c}\\ \vdots\\ \\ \vdots\\ \\ R_{lc}\\ \end{matrix}\right)\leq\left(\begin{matrix}I_{1,00...0}\\ I_{1,00...1}\\ \vdots\\ I_{1,11...1}\\ I_{2,00...0}\\ I_{2,00...1}\\ \vdots\\ I_{2,11...1}\\ \vdots\\ \vdots\\ I_{l,00...0}\\ \\ \\ I_{l,11...1}\\ \end{matrix}\right) (2)

If we denote A as the matrix whose rows are all possible binary l𝑙l-tuples starting from (0, 0, …, 0) in an increasing order, then Aisubscript𝐴𝑖A_{i} is the matrix formed by multiplying the i𝑖ith column in A by -1. We show the rate coefficients matrix in (LABEL:eqq) with Clsubscript𝐶𝑙C_{l}. We show the rate vector with R¯¯tsuperscript¯¯𝑅𝑡\underline{\underline{R}}^{t}, and the mutual information vector with Ilsubscript𝐼𝑙I_{l}, so we may write (LABEL:eqq) in compact form as ClR¯¯tIlsubscript𝐶𝑙superscript¯¯𝑅𝑡subscript𝐼𝑙C_{l}\underline{\underline{R}}^{t}\leq I_{l}. To clarify the method of forming the above mentioned inequations, we give the following example, which illustrates the inequations for a two sender-receiver interference channel:

C2R¯¯tI2(1010101001010101|0001101100011011)(R1R2R1cR2c)(I1,00I1,01I1,10I1,11I2,00I2,01I2,10I2,11)subscript𝐶2superscript¯¯𝑅𝑡subscript𝐼2conditionalmatrix1010101001010101matrix0001101100011011matrixsubscript𝑅1subscript𝑅2subscript𝑅1𝑐subscript𝑅2𝑐matrixsubscript𝐼100subscript𝐼101subscript𝐼110subscript𝐼111subscript𝐼200subscript𝐼201subscript𝐼210subscript𝐼211C_{2}\underline{\underline{R}}^{t}\leq I_{2}\Rightarrow\left(\begin{matrix}1&0\\ 1&0\\ 1&0\\ 1&0\\ 0&1\\ 0&1\\ 0&1\\ 0&1\\ \end{matrix}\right|\left.\begin{matrix}0&0\\ 0&1\\ -1&0\\ -1&1\\ 0&0\\ 0&-1\\ 1&0\\ 1&-1\\ \end{matrix}\right)\left(\begin{matrix}R_{1}\\ R_{2}\\ R_{1c}\\ R_{2c}\\ \end{matrix}\right)\leq\left(\begin{matrix}I_{1,00}\\ I_{1,01}\\ I_{1,10}\\ I_{1,11}\\ I_{2,00}\\ I_{2,01}\\ I_{2,10}\\ I_{2,11}\\ \end{matrix}\right)

Note that these are the same inequations given in [12]. To solve this system with the conventional FME method, one must eliminate all of the variables Ric,i[1:l]R_{ic},i\in[1:l]; this process takes l𝑙l rounds of applying FME to the matrix. According to [12], the size of the matrix grows at a worst case rate of k24superscript𝑘24\frac{k^{2}}{4} each round. This complexity renders the aforementioned method impractical for l𝑙l larger than 2, since, after applying FME, one must eliminate the redundant inequations in the resulting matrix using an exhaustive search. Finding the redundant inequalities often needs more time than eliminating the auxiliary variables, which itself is a time-consuming process.

IV Our New Method

Here, we present a new method for solving (LABEL:eqq) which not only requires much less calculation in comparison with the FME but also produces no redundant inequations assumning the mutual-informations on the right side of the inequations are independant. Before proceeding with the solution, we present some definitions. We show the matrixes resulting from l𝑙l rounds of FME by Clsubscriptsuperscript𝐶𝑙C^{*}_{l} and Ilsubscriptsuperscript𝐼𝑙I^{*}_{l}, so the resulting inequations can be written in the form ClR¯¯tIlsubscriptsuperscript𝐶𝑙superscript¯¯𝑅𝑡subscriptsuperscript𝐼𝑙C^{*}_{l}\underline{\underline{R}}^{t}\leq I^{*}_{l}. Note that the second l𝑙l columns of Clsubscriptsuperscript𝐶𝑙C^{*}_{l} are zero. It is also clear that if we show the rows of Clsubscript𝐶𝑙C_{l} by Cl(l,α1,α2,,αl)subscript𝐶𝑙𝑙subscript𝛼1subscript𝛼2subscript𝛼𝑙C_{l(l,\alpha_{1},\alpha_{2},...,\alpha_{l})}, then all rows of Clsubscriptsuperscript𝐶𝑙C^{*}_{l} and Ilsubscriptsuperscript𝐼𝑙I^{*}_{l} can be shown by

Cl(k)=i,α1,α2,,αlak(i,α1,α2,,αl)Cl(i,α1,α2,,αl),subscriptsuperscript𝐶𝑙𝑘subscript𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝑎𝑘𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝐶𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙C^{*}_{l(k)}=\sum_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}a_{k(i,\alpha_{1},\alpha_{2},...,\alpha_{l})}C_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})},
Il(k)=i,α1,α2,,αlak(i,α1,α2,,αl)Il(i,α1,α2,,αl),subscriptsuperscript𝐼𝑙𝑘subscript𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝑎𝑘𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝐼𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙I^{*}_{l(k)}=\sum_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}a_{k(i,\alpha_{1},\alpha_{2},...,\alpha_{l})}I_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})},

where

αi{0,1},i[1:l],ak(i,α1,α2,,αl)0\alpha_{i}\in\{0,1\},i\in[1:l],a_{k(i,\alpha_{1},\alpha_{2},...,\alpha_{l})}\geq 0

Let GClsubscript𝐺subscript𝐶𝑙G_{C_{l}} be the solution space of the inequations in (LABEL:eqq) for rates R1,R2,,Rlsubscript𝑅1subscript𝑅2subscript𝑅𝑙R_{1},R_{2},...,R_{l}, and let GClsubscript𝐺subscriptsuperscript𝐶𝑙G_{C^{*}_{l}} be the solution space of ClR¯¯tIlsubscriptsuperscript𝐶𝑙superscript¯¯𝑅𝑡subscriptsuperscript𝐼𝑙C^{*}_{l}\underline{\underline{R}}^{t}\leq I^{*}_{l}. Furthermore, we define the matrix D as the matrix whose rows are formed by all of the linear combinations of Cl(i,α1,α2,,αl)subscript𝐶𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙C_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})} with positive coefficients and with the property that the second l𝑙l columns of D are zero. In other words, D¯¯¯¯𝐷\underline{\underline{D}} is a row of D iff

D¯¯=i,α1,α2,,αlai,α1,α2,,αlCl(i,α1,α2,,αl)¯¯𝐷subscript𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝑎𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝐶𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙\underline{\underline{D}}=\sum_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}a_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}C_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})} (3)

with the following conditions:

ak(i,α1,α2,,αl)0subscript𝑎𝑘𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙0a_{k(i,\alpha_{1},\alpha_{2},...,\alpha_{l})}\geq 0
Dl+12l=0;superscriptsubscript𝐷𝑙12𝑙0D_{l+1}^{2l}=0; (4)

We define GDlsubscript𝐺subscript𝐷𝑙G_{D_{l}} as the solution space of the following inequations:

DlR¯¯tIDlsubscript𝐷𝑙superscript¯¯𝑅𝑡subscript𝐼subscript𝐷𝑙D_{l}\underline{\underline{R}}^{t}\leq I_{D_{l}} (5)

where

IDl=i,α1,α2,,αlai,α1,α2,,αlIl(i,α1,α2,,αl)subscript𝐼subscript𝐷𝑙subscript𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝑎𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝐼𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙I_{D_{l}}=\sum_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}a_{i,\alpha_{1},\alpha_{2},...,\alpha_{l}}I_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})}
Theorem 1

GClsubscript𝐺subscriptsuperscript𝐶𝑙G_{C^{*}_{l}} is equal to GDlsubscript𝐺subscript𝐷𝑙G_{D_{l}}.

Proof:

The proof is straightforward. Since all rows of Dlsubscript𝐷𝑙D_{l} are linear combinations of rows of Clsubscript𝐶𝑙C_{l} with positive coefficients, all inequations in (5) result from inequations in (LABEL:eqq). Hence, we have GClGDlsubscript𝐺subscript𝐶𝑙subscript𝐺subscript𝐷𝑙G_{C_{l}}\subseteq G_{D_{l}}, and by the FME, we know that GClsubscript𝐺subscript𝐶𝑙G_{C_{l}} is equal to GClsubscript𝐺subscriptsuperscript𝐶𝑙G_{C^{*}_{l}} for (R1,R2,,Rl)subscript𝑅1subscript𝑅2subscript𝑅𝑙(R_{1},R_{2},...,R_{l}), we have GClGDlsubscript𝐺subscriptsuperscript𝐶𝑙subscript𝐺subscript𝐷𝑙G_{C^{*}_{l}}\subseteq G_{D_{l}}. On the other hand, since all the rows of Clsubscriptsuperscript𝐶𝑙C^{*}_{l} are linear combinations of rows of Clsubscript𝐶𝑙C_{l} with the property that for each row C¯¯superscript¯¯𝐶\underline{\underline{C}}^{*}, Cl+12l=0subscriptsuperscriptsuperscript𝐶2𝑙𝑙10{C^{*}}^{2l}_{l+1}=0, any row C¯¯superscript¯¯𝐶\underline{\underline{C}}^{*} of Clsubscriptsuperscript𝐶𝑙C^{*}_{l} is a row of Dlsubscript𝐷𝑙D_{l}, and so GDlGClsubscript𝐺subscript𝐷𝑙subscript𝐺subscriptsuperscript𝐶𝑙G_{D_{l}}\subseteq G_{C^{*}_{l}} and the proof is complete. ∎

Now, we propose that instead of finding GClsubscript𝐺subscriptsuperscript𝐶𝑙G_{C^{*}_{l}}, we try to find GDlsubscript𝐺subscript𝐷𝑙G_{D_{l}}. From (4), we know that for every D in (3) we have

i,α1α2,,αj1,αj+1,,αlijai,α1,α2,,αj1,1,αj+1,,αlsubscript𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑗1subscript𝛼𝑗1subscript𝛼𝑙𝑖𝑗subscript𝑎𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑗11subscript𝛼𝑗1subscript𝛼𝑙\displaystyle\sum_{\begin{subarray}{c}i,\alpha_{1}\alpha_{2},...,\alpha_{j-1},\alpha_{j+1},...,\alpha_{l}\\ i\neq j\end{subarray}}a_{i,\alpha_{1},\alpha_{2},...,\alpha_{j-1},1,\alpha_{j+1},...,\alpha_{l}}
=α1,α2,,αj1,αj+1,,αlaj,α1,α2,,αj1,1,αj+1,,αlabsentsubscriptsubscript𝛼1subscript𝛼2subscript𝛼𝑗1subscript𝛼𝑗1subscript𝛼𝑙subscript𝑎𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑗11subscript𝛼𝑗1subscript𝛼𝑙\displaystyle=\sum_{\alpha_{1},\alpha_{2},...,\alpha_{j-1},\alpha_{j+1},...,\alpha_{l}}a_{j,\alpha_{1},\alpha_{2},...,\alpha_{j-1},1,\alpha_{j+1},...,\alpha_{l}} (6)

Now we need to choose inequations in (5) whose linear combinations with positive coefficients produce all inequations in (5). Without loss of generality, we consider aj,α1,α2,,αlZ+subscript𝑎𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑙superscript𝑍a_{j,\alpha_{1},\alpha_{2},...,\alpha_{l}}\in Z^{+}. We give the following definition as in [13];
Definition: We call a set of vectors H, generator vectors of S, if every vector in S can be expressed as a linear combination of vectors in H with positive coefficients and no vector in H may be expressed in such a way by other vectors in H. Such a set of vectors is called the Hilbert Basis of the vector space S.

The problem of finding a Hilbert Basis for the solutions of a Diophantine system of equations such as (6) has many solutions. One solution is to use an algorithm called Normaliz introduced by Koch in [5]. We use this method to solve (6). If the set of aj,α1,α2,,αlsubscript𝑎𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑙a_{j,\alpha_{1},\alpha_{2},...,\alpha_{l}}, j=1,,l𝑗1𝑙j=1,...,l,αi{0,1}subscript𝛼𝑖01\alpha_{i}\in\{0,1\} is an answer for (6), then we have the following inequation for R1,,Rlsubscript𝑅1subscript𝑅𝑙R_{1},...,R_{l}:

j=1l((αiaj,α1,α2,,αl)Rj)αi,jaj,α1,α2,,αlIj,α1,α2,,αlsuperscriptsubscript𝑗1𝑙subscriptsubscript𝛼𝑖subscript𝑎𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝑅𝑗subscriptsubscript𝛼𝑖𝑗subscript𝑎𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑙subscript𝐼𝑗subscript𝛼1subscript𝛼2subscript𝛼𝑙\sum_{j=1}^{l}((\sum_{\alpha_{i}}a_{j,\alpha_{1},\alpha_{2},...,\alpha_{l}})R_{j})\leq\sum_{\alpha_{i},j}a_{j,\alpha_{1},\alpha_{2},...,\alpha_{l}}I_{j,\alpha_{1},\alpha_{2},...,\alpha_{l}}

Since Normaliz provides a Hilbert Basis solution for the equations in (6), and since each solution for (6) represents a unique equation in (5) and adding two answers from (6) with positive coefficients is equivalent to adding two inequations in (5) with positive coefficients, the resulting inequations contain all the non-redundant inequations in (5). Hence, by applying Normaliz, we can calculate the solution to our original problem. Since the dimension of the Diophantine space in (6) is much greater than the number of constraints we have, it is better to use Normaliz Dual as suggested in [6]. Our results for the two sender-receiver IC are the same as HK rate region in [10].

To this point, we have found the rate region by eliminating the auxiliary variables. Since, in the case of the above channel, all the mutual-informations Il(i,α1,α2,,αl)subscript𝐼𝑙𝑖subscript𝛼1subscript𝛼2subscript𝛼𝑙I_{l(i,\alpha_{1},\alpha_{2},...,\alpha_{l})} are independent of each other (i.e. they don’t have a clear relation with each other), there are no redundant inequations in our final system of inequations and so by the Caratheodory theorem [14] Q is bounded by the number of resulting inequations after applying Normaliz and finding the dual system.

V Generalization to Other Problems

The method proposed above may be used in any other problem which could be solved using the FME. The following algorithm is used:
1- Define the encoding and decoding schemes.
2- Find the inequations matrix corresponding to the defined schemes.
3- Write the Diophantine equations of the matrix.
4- Use Normaliz to solve the Diophantine problem.
5- For each answer vector for the Diophantine problems, write the corresponding inequation.

VI Simulation and Results

Here, we use our method to solve some problems which have been solved using the FME method previously. The simulation results are illustrated in (I):

TABLE I: Results of Applying Our New Method on Some Examples
Name Number of Number of Basis Non-Redundant t
constraints aux. variables Elements Elements (sec)
HK 8 2 7 7 1sabsent1𝑠\leq 1s
l=2
HK 24 3 153 153 1sabsent1𝑠\leq 1s
l=3
HK 64 4 56384 56384 203s
l=4
Ref. 15 29 6 286 60 \leq1s
Ref. 16 12 5 6 3 \leq1s

The results are obtained from running Normaliz 2.2 on a 2.4 i5 CPU. The 4th column indicates the number of inequations resulting from applying our method while the 5th column indicates the number of non-redundant constraints. Note that in cases such as [15] and [16] where there are certain relations between mutual-informations on the right side of the inequations, there may be redundant answers in our method. However all these redundant answers will be present in the FME case as well. Table (I) shows the efficiency of our method in comparison to the FME method. There are several features of our method that make it more efficient in comparison with the former method. One feature is that the FME method eliminates one variable at a time, while our method eliminates all variables in one step. Another feature is that our method is much faster compared to the previous method. Another important advantage of the new method is that it doesn’t introduce many of the redundant inequation present in the FME solution. Note that one of the most time-consuming parts of the FME method is the removal of inequations which are linear combinations of previous inequations with positive coefficients. This process uses an exhaustive search and is very inefficient. Our method doesn’t introduce any such redundant answers.

VII Conclusion

In this paper, we introduced a new method for performing variable elimination in systems of inequations. We illustrated the implementation of our new method on a special case of IC. A generalization of the method was presented and in the end some simulation results were compared with the actual results after applying FME and removing redundant answers. It was shown that our new method is much faster and more efficient than the FME method. Our method eliminates all variables in one step while the FME method uses a step by step approach. Also, our method produces fewer redundant inequations in comparison to the FME method.

Acknowledgments

We would like to express our appreciation for the ISSL Group of Sharif University of Technology and specially B. Akhbari and M. Mirmohseni for their valuable comments.

References

  • [1] J-B.J. Fourier, reported in: Analyse des travaux de l‘Acadamie Royale des Sciences, pendant l‘annee 1824, Partie mathematique, Histoire de l‘Academie Royale des Sciences de l‘Institut de France 7 (1827) xlvii lv. (Partial English translation in: D.A. Kohler, Translation of a Report by Fourier on his work on Linear Inequalities, Opsearch 10 (1973) 38 42.)
  • [2] T.S. Motzkin. Beiträge zur Theorie der linearen Ungleichungen. (Inangural Dissertation Basel). Azriel, Jerusalem, 1936. English translation: Contributions to the theory of linear inequalities, RAND Corporation Translation 22, Santa Monica, CA, 1952. Reprinted in Theodore S. Motzkin: Selected Papers (D. Cantor, B. Gordon, B. Rothschild. eds.), Birkhäuser, Boston, 1983, pp. 1-80.
  • [3] G.B. Dantzig, and B.C. Eaves, Fourier-Motzkin Elimination and its Dual,.   Journal of Combinatorial Theory Ser. A, 14:288 297, 1973.
  • [4] C. W. Kebler, Parallel fourier-motzkin elimination, In Euro-Pat, Vol. II, pp. 66 71, 1996.
  • [5] R. Koch, Affine Monoids, Hilbert Bases and Hilbert Functions. PhD thesis, Osnabr ck university, Germany, 2003.
  • [6] W. Bruns and B. Ichim. Normaliz: algorithms for affine monoids and rational cones, J. Algebra 324 (2010), no. 5, 1098 1113.
  • [7] C. E. Shannon, Two-way communication channels, in Proc. 4th Berkeley Symp. on Mathematical Statistics and Probability, vol. 1, Berkeley, CA: Univ. California Press, 1961.
  • [8] R. Ahlswede, The capacity region of a channel with two senders and two receivers, Annals Probabil., vol. 2, no. 5, pp. 805-814, 1974.
  • [9] A. B. Carleial, Interference channels, IEEE Trans. Inform. Theory, vol. IT-24, no. 1, pp. 60 70, Jan. 1978.
  • [10] T. S. Han and K. Kobayashi, A new achievable rate region for the interference channel,.   IEEE Trans. Inf. Theory, vol. 27, no. 1, pp. 49-60, 1981.
  • [11] R. Etkin, D. Tse, and H. Wang, Gaussian interference channel capacity to within one bit, IEEE Trans. Inf. Theory, vol. 54, no. 12, pp. 5534 5562, 2008.
  • [12] A. El Gamal and Y.-H. Kim,Lecture notes on network information theory .   http://arxiv.org/abs/1001.3404, Jan. 2010.
  • [13] W. Bruns and J. Gubeladze. Polytopes, rings, and K-theory. Springer Monographs in Mathematics, 2009.
  • [14] C. Carath odory, ber den Variabilit tsbereich der Fourierschen Konstanten von positiven harmonischen Funktionen, Rend. Circ. Mat. Palermo, Vol. 32, pp. 193-217, 1911.
  • [15] M. Mirmohseni, B. Akhbari, and M. R. Aref, On the capacity of causal cognitive interference channel with delay, submitted to IEEE Trans. Inform. Theory, April 2010.
  • [16] Y. K. Chia, and A. El Gamal, 3-Receiver broadcast channels with common and confidential messages, IEEE Int. Symp. Information Theory (ISIT 2010), Seoul, Korea, pp. 1849 - 1853, June 28-July 3, 2009.