A New Method for Variable Elimination in Systems of Inequations
Abstract
In this paper, we present a new method for variable elimination in systems of inequations which is much faster than the Fourier-Motzkin Elimination (FME) method. In our method, a linear Diophantine problem is introduced which is dual to our original problem. The new Diophantine system is then solved, and the final result is calculated by finding the dual inequations system. Our new method uses the algorithm Normaliz to find the Hilbert basis of the solution space of the given Diophantine problem. We introduce a problem in the interference channel with multiple nodes and solve it with our new method. Next, we generalize our method to all problems involving FME and in the end we compare our method with the previous method. We show that our method has many advantages in comparison to the previous method. It does not produce many of the redundant answers of the FME method. It also solves the whole problem in one step whereas the previous method uses a step by step approach in eliminating each auxiliary variable.
I Introduction
The FME Method was first introduced by Fourier [1] in 1827 and was rediscovered in Motzkin’s thesis in 1936 [2]. Considering the set of inequations the method provides a scheme for determining the existence of a real or integer solution to the given problem. The method uses a step by step algorithm, in which each step reduces the dimension of the answer space by projecting the answer space on a hyperplane. The procedure continues until the dimension is reduced to one; if the one dimensional answer is feasible, then the solution exists. A dual mode for FME was introduced in 1972 by Dantzik [3].
The FME method is widely used in solving rate region problems since there is often the need to eliminate the auxiliary variables introduced into the problem from the resulting inequations and hence reduce the dimension of the solution space by finding its projection on a given plane. The complexity of calculation in each step is of the order of , with being the number of constraints in the system of inequalities. This complexity makes the method unfeasible for high dimensional problems. There have been many attempts to decrease the calculation time of the method. In 1996, Kebler [4] introduced a parallel computation method which was more efficient when solving the elimination problem on a computer.
In this paper, we present a new method for eliminating auxiliary variables in inequations which is much faster than the FME method. In our method, we first introduce a linear Diophantine problem which is dual to the original problem. We then proceed by solving this new Diophantine system, next we calculate the final result by finding the dual inequations system. Our new method uses the algorithm Normaliz to find the Hilbert basis of the solution space of the given Diophantine problem.
Normaliz was first introduced as a program in 1997 by R. Koch. In 2003, the algorithm was presented in Koch’s thesis [5]. Normaliz was originally introduced to compute the lattice points in a lattice polytope and the integral closure of a monomial ideal. Koch mentions in his thesis that the algorithm may be extended to find the Hilbert basis of the solution space of a system of linear Diophantine equations. However, applying the algorithm introduced in [5] to information theoritic problems is still time-consuming. In 2010, Bruns and Ichim [6] introduced a dual algorithm which is much faster when applied to problems with a greater number of constraints than auxiliary variables. Since this is the case in almost all information theory problems, we use this method to find the desired Hilbert Basis solution.
One of the most important fundamental channels in communication systems is the Interference Channel (IC). The IC was initiated by Shannon [7]. Simple outer bounds for the IC were established by Ahlswede [8]. Later, Carlieal [9], using superposition encoding and sequential decoding, characterized an achievable rate region for general discrete memoryless IC. In the IC each receiver is only interested in the message from its corresponding sender. However, it might decode messages from other senders to reduce the interference effect. Han and Kobayashi (HK) utilized rate splitting techniques in [10], and applied simultaneous superposition coding techniques to establish the largest inner bound introduced for the general IC to date. Computing the capacity region of general IC is still an open problem, but in special cases such as strong interference regime the capacity region has been established, and the region was the same as the HK rate region [11]. In the method used to achieve the HK rate region, the messages are split into two parts, the private and common parts; private messages are decoded by the corresponding receiver, while the common messages are decoded by all receivers.
Rate splitting has become a prominent tool for solving rate region problems. However, there is a drawback to the method in that solving the channel with the new split messages yields a system of inequations based on the split-rates. Since we are interested in each sender’s total rates, the auxiliary variables must be eliminated. The FME method is needed to eliminate these auxiliary rates.
The rest of the paper is organized as follows: In Section II, we first introduce our model of an transmitter-receiver interference channel. In Section III, we introduce a problem in the IC. In this section, we use some simplifying assumptions, since the goal of the section is to explain the new elimination method. In Section IV, we solve the problem with our new method. In Section V, we present a generalized algorithm for our new method which could be applied to any problem involving FME. In Section VI, we present results of applying our new method to several problems which were solved previously using the FME method. Section VII concludes the paper.
II Channel Moddel
In this section, we introduce our model of IC. We denote random variables by upper case letters e.g. X, their realizations are presented by lower case letters e.g. x in finite sets denoted by . We use to denote the vector . We denote if i=1, while for i=1 and k=, we show the vector by . Also, if i=1 and k=, we show the vector by x. Furthermore is the compact form of where .
A discrete Interference Channel, with senders and receivers is the 2+1 tuple where are the input alphabets, are the output alphabets for i=1,2,…, and is the conditional probability of the channel. Each decoder is only interested in decoding its corresponding sender’s message; however, in order to reduce the interference effect, receivers might decode the message from other senders as well. The channel is memoryless and time invariant in the sense that
A code consists of (i) message sets , (ii) encoding functions , (iii) decoding functions where . We define the average probability of error for this code as . A nonnegative rate -tuple is said to be achievable if there exists a code for which as tends to infinity.
III Problem Statement
In this section, we present our coding strategy which is a generalization of the method used in [10]. We describe the problem in solving the resulting inequations with the conventional FME method. After stating the problem, we introduce our method and compare the results with the former procedure.
Since our goal in this paper is to introduce an alternative elimination method for the FME method, we use some simplifying assumptions in the channel. We consider the channel to be symmetric in the sense that the th sender sends the same amount of information to all receivers other than its own corresponding receiver. This allows each sender to send the same common message to all receivers. We split each message into common and private parts . The common part of the message is decoded by all receivers while the private part is decoded only by the th receiver. Another suitable assumption would have been to assume a sense of degradedness between the receivers in the following way: assume that receiver receives all the common massages sent by sender ; receiver receives all the common messages received by the receivers other than , and so on. In this case we had to split the message in each sender to common parts and one private part in the following way: split the message into ). is decoded by the receiver with the least information about the massage from sender ; the reciever with the second least information about this message decodes , and so on. The rest of the solution would stay the same. Clearly, the symmetric case mentioned above is a special case of this latter case. Since the two cases have the same solution, and for the sake of simplicity, we solve the channel for the symmetric case in this paper.
Codebook Generation:
Let , where Q is the time-sharing variable. Let denote the set of all jont p.m.fs on , which can be written in the form
For any defined above, generate , i.i.d where . For each , generate , i.i.d . The time-sharing variable has no impact on the rest of the proof and for the sake of brevity, we ignore it until the end of the proof where we bound its cardinality.
Encoding:
At the beginning of each block, the th sender picks related according to and sends it.
Decoding:
At the end of each block, the th receiver finds such that:
Without loss of generality we consider the case where all senders have sent index 1 as their messages. We investigate all possible scenarios in the first decoder. The decoder must decode messages , the decoding is errorless if . Using the same method as in [12] and by the AEP rule, it is clear that the probability of error approaches zero as (e.g. rate -tuple is achievable) iff the rates satisfy the following set of equations:
(1) |
where
and if ; also, for , we have . We note that , substituting into (1) we have
These are the inequations for decoder 1. We rewrite the inequations for all decoders in matrix form:
(2) |
If we denote A as the matrix whose rows are all possible binary -tuples starting from (0, 0, …, 0) in an increasing order, then is the matrix formed by multiplying the th column in A by -1. We show the rate coefficients matrix in (LABEL:eqq) with . We show the rate vector with , and the mutual information vector with , so we may write (LABEL:eqq) in compact form as . To clarify the method of forming the above mentioned inequations, we give the following example, which illustrates the inequations for a two sender-receiver interference channel:
Note that these are the same inequations given in [12]. To solve this system with the conventional FME method, one must eliminate all of the variables ; this process takes rounds of applying FME to the matrix. According to [12], the size of the matrix grows at a worst case rate of each round. This complexity renders the aforementioned method impractical for larger than 2, since, after applying FME, one must eliminate the redundant inequations in the resulting matrix using an exhaustive search. Finding the redundant inequalities often needs more time than eliminating the auxiliary variables, which itself is a time-consuming process.
IV Our New Method
Here, we present a new method for solving (LABEL:eqq) which not only requires much less calculation in comparison with the FME but also produces no redundant inequations assumning the mutual-informations on the right side of the inequations are independant. Before proceeding with the solution, we present some definitions. We show the matrixes resulting from rounds of FME by and , so the resulting inequations can be written in the form . Note that the second columns of are zero. It is also clear that if we show the rows of by , then all rows of and can be shown by
where
Let be the solution space of the inequations in (LABEL:eqq) for rates , and let be the solution space of . Furthermore, we define the matrix D as the matrix whose rows are formed by all of the linear combinations of with positive coefficients and with the property that the second columns of D are zero. In other words, is a row of D iff
(3) |
with the following conditions:
(4) |
We define as the solution space of the following inequations:
(5) |
where
Theorem 1
is equal to .
Proof:
The proof is straightforward. Since all rows of are linear combinations of rows of with positive coefficients, all inequations in (5) result from inequations in (LABEL:eqq). Hence, we have , and by the FME, we know that is equal to for , we have . On the other hand, since all the rows of are linear combinations of rows of with the property that for each row , , any row of is a row of , and so and the proof is complete. ∎
Now, we propose that instead of finding , we try to find . From (4), we know that for every D in (3) we have
(6) |
Now we need to choose inequations in (5) whose linear combinations with positive coefficients produce all inequations in (5). Without loss of generality, we consider . We give the following definition as in [13];
Definition: We call a set of vectors H, generator vectors of S, if every vector in S can be expressed as a linear combination of vectors in H with positive coefficients and no vector in H may be expressed in such a way by other vectors in H. Such a set of vectors is called the Hilbert Basis of the vector space S.
The problem of finding a Hilbert Basis for the solutions of a Diophantine system of equations such as (6) has many solutions. One solution is to use an algorithm called Normaliz introduced by Koch in [5]. We use this method to solve (6). If the set of , , is an answer for (6), then we have the following inequation for :
Since Normaliz provides a Hilbert Basis solution for the equations in (6), and since each solution for (6) represents a unique equation in (5) and adding two answers from (6) with positive coefficients is equivalent to adding two inequations in (5) with positive coefficients, the resulting inequations contain all the non-redundant inequations in (5). Hence, by applying Normaliz, we can calculate the solution to our original problem. Since the dimension of the Diophantine space in (6) is much greater than the number of constraints we have, it is better to use Normaliz Dual as suggested in [6]. Our results for the two sender-receiver IC are the same as HK rate region in [10].
To this point, we have found the rate region by eliminating the auxiliary variables. Since, in the case of the above channel, all the mutual-informations are independent of each other (i.e. they don’t have a clear relation with each other), there are no redundant inequations in our final system of inequations and so by the Caratheodory theorem [14] Q is bounded by the number of resulting inequations after applying Normaliz and finding the dual system.
V Generalization to Other Problems
The method proposed above may be used in any other problem which could be solved using the FME. The following algorithm is used:
1- Define the encoding and decoding schemes.
2- Find the inequations matrix corresponding to the defined schemes.
3- Write the Diophantine equations of the matrix.
4- Use Normaliz to solve the Diophantine problem.
5- For each answer vector for the Diophantine problems, write the corresponding inequation.
VI Simulation and Results
Here, we use our method to solve some problems which have been solved using the FME method previously. The simulation results are illustrated in (I):
Name | Number of | Number of | Basis | Non-Redundant | t |
constraints | aux. variables | Elements | Elements | (sec) | |
HK | 8 | 2 | 7 | 7 | |
l=2 | |||||
HK | 24 | 3 | 153 | 153 | |
l=3 | |||||
HK | 64 | 4 | 56384 | 56384 | 203s |
l=4 | |||||
Ref. 15 | 29 | 6 | 286 | 60 | 1s |
Ref. 16 | 12 | 5 | 6 | 3 | 1s |
The results are obtained from running Normaliz 2.2 on a 2.4 i5 CPU. The 4th column indicates the number of inequations resulting from applying our method while the 5th column indicates the number of non-redundant constraints. Note that in cases such as [15] and [16] where there are certain relations between mutual-informations on the right side of the inequations, there may be redundant answers in our method. However all these redundant answers will be present in the FME case as well. Table (I) shows the efficiency of our method in comparison to the FME method. There are several features of our method that make it more efficient in comparison with the former method. One feature is that the FME method eliminates one variable at a time, while our method eliminates all variables in one step. Another feature is that our method is much faster compared to the previous method. Another important advantage of the new method is that it doesn’t introduce many of the redundant inequation present in the FME solution. Note that one of the most time-consuming parts of the FME method is the removal of inequations which are linear combinations of previous inequations with positive coefficients. This process uses an exhaustive search and is very inefficient. Our method doesn’t introduce any such redundant answers.
VII Conclusion
In this paper, we introduced a new method for performing variable elimination in systems of inequations. We illustrated the implementation of our new method on a special case of IC. A generalization of the method was presented and in the end some simulation results were compared with the actual results after applying FME and removing redundant answers. It was shown that our new method is much faster and more efficient than the FME method. Our method eliminates all variables in one step while the FME method uses a step by step approach. Also, our method produces fewer redundant inequations in comparison to the FME method.
Acknowledgments
We would like to express our appreciation for the ISSL Group of Sharif University of Technology and specially B. Akhbari and M. Mirmohseni for their valuable comments.
References
- [1] J-B.J. Fourier, reported in: Analyse des travaux de l‘Acadamie Royale des Sciences, pendant l‘annee 1824, Partie mathematique, Histoire de l‘Academie Royale des Sciences de l‘Institut de France 7 (1827) xlvii lv. (Partial English translation in: D.A. Kohler, Translation of a Report by Fourier on his work on Linear Inequalities, Opsearch 10 (1973) 38 42.)
- [2] T.S. Motzkin. Beiträge zur Theorie der linearen Ungleichungen. (Inangural Dissertation Basel). Azriel, Jerusalem, 1936. English translation: Contributions to the theory of linear inequalities, RAND Corporation Translation 22, Santa Monica, CA, 1952. Reprinted in Theodore S. Motzkin: Selected Papers (D. Cantor, B. Gordon, B. Rothschild. eds.), Birkhäuser, Boston, 1983, pp. 1-80.
- [3] G.B. Dantzig, and B.C. Eaves, Fourier-Motzkin Elimination and its Dual,. Journal of Combinatorial Theory Ser. A, 14:288 297, 1973.
- [4] C. W. Kebler, Parallel fourier-motzkin elimination, In Euro-Pat, Vol. II, pp. 66 71, 1996.
- [5] R. Koch, Affine Monoids, Hilbert Bases and Hilbert Functions. PhD thesis, Osnabr ck university, Germany, 2003.
- [6] W. Bruns and B. Ichim. Normaliz: algorithms for affine monoids and rational cones, J. Algebra 324 (2010), no. 5, 1098 1113.
- [7] C. E. Shannon, Two-way communication channels, in Proc. 4th Berkeley Symp. on Mathematical Statistics and Probability, vol. 1, Berkeley, CA: Univ. California Press, 1961.
- [8] R. Ahlswede, The capacity region of a channel with two senders and two receivers, Annals Probabil., vol. 2, no. 5, pp. 805-814, 1974.
- [9] A. B. Carleial, Interference channels, IEEE Trans. Inform. Theory, vol. IT-24, no. 1, pp. 60 70, Jan. 1978.
- [10] T. S. Han and K. Kobayashi, A new achievable rate region for the interference channel,. IEEE Trans. Inf. Theory, vol. 27, no. 1, pp. 49-60, 1981.
- [11] R. Etkin, D. Tse, and H. Wang, Gaussian interference channel capacity to within one bit, IEEE Trans. Inf. Theory, vol. 54, no. 12, pp. 5534 5562, 2008.
- [12] A. El Gamal and Y.-H. Kim,Lecture notes on network information theory . http://arxiv.org/abs/1001.3404, Jan. 2010.
- [13] W. Bruns and J. Gubeladze. Polytopes, rings, and K-theory. Springer Monographs in Mathematics, 2009.
- [14] C. Carath odory, ber den Variabilit tsbereich der Fourierschen Konstanten von positiven harmonischen Funktionen, Rend. Circ. Mat. Palermo, Vol. 32, pp. 193-217, 1911.
- [15] M. Mirmohseni, B. Akhbari, and M. R. Aref, On the capacity of causal cognitive interference channel with delay, submitted to IEEE Trans. Inform. Theory, April 2010.
- [16] Y. K. Chia, and A. El Gamal, 3-Receiver broadcast channels with common and confidential messages, IEEE Int. Symp. Information Theory (ISIT 2010), Seoul, Korea, pp. 1849 - 1853, June 28-July 3, 2009.